11:21:17,545 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:17,555 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3549068605584841628] 11:21:17,556 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:17,557 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:17,557 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:17,569 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:17,569 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:17,571 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:17,580 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:17,580 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:17,596 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:21:17,607 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [740101089859104711] 11:21:17,607 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:17,609 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:17,609 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:17,615 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:17,615 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:17,644 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:17,715 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4811246403688835085] 11:21:17,715 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:17,723 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:17,723 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:17,727 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:17,736 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:17,738 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:17,738 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:17,738 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:17,744 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 11:21:17,746 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:17,751 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1078318649] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,751 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 11:21:17,761 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 11:21:17,761 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 11:21:17,761 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:17,761 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,761 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7579d2a2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4630b37d 11:21:17,761 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,761 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:17,761 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,761 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,764 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@72827bef 11:21:17,764 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709032877761, None)) 11:21:17,764 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032877760,0,0)) 11:21:17,764 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4630b37d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@604cd724 11:21:17,765 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@67375368 11:21:17,765 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 11:21:17,765 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:17,767 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:17,769 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 11:21:17,770 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:17,770 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563] 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964] 11:21:17,775 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 11:21:17,775 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,776 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-483953141 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,776 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-483953141] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,776 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1904142823] 11:21:17,776 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1904142823] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,776 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 11:21:17,777 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 11:21:17,777 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#423754548 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,777 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1727897468] 11:21:17,783 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 11:21:17,786 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:17,786 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:17,786 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 11:21:17,786 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 11:21:17,786 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:17,786 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:17,790 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 11:21:17,792 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1715249497] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,797 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,797 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#423754548] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,797 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1727897468] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:17,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 11:21:17,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 11:21:17,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 11:21:17,798 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 11:21:17,798 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:21:17,798 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:17,798 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 11:21:17,798 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 11:21:17,798 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 11:21:17,798 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:21:17,798 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 11:21:17,801 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 11:21:17,801 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 11:21:17,801 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:17,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:17,801 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@19c6741a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@34def7fc 11:21:17,801 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6801801b 11:21:17,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 11:21:17,802 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709032877801, None)) 11:21:17,802 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:17,802 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,802 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,802 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032877800,0,0)) 11:21:17,802 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@34def7fc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@c0b49d9 11:21:17,802 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 11:21:17,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4c6734c9 11:21:17,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:17,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:17,809 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,809 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:17,809 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1031296669] 11:21:17,809 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 11:21:17,809 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 11:21:17,810 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 11:21:17,810 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 11:21:17,810 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 11:21:17,810 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1031296669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,810 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#857776843] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,810 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-923321908 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:17,810 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-986434214] 11:21:17,811 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-923321908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,811 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-986434214] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,811 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1794452892] 11:21:17,811 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1763939235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,811 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1794452892] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:17,812 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,812 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:21:17,812 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#857776843] 11:21:17,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:17,813 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:17,813 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 11:21:17,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:17,813 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 11:21:17,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 11:21:17,813 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:17,815 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1763939235 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 11:21:17,815 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 11:21:17,815 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:21:17,816 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 11:21:17,818 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:17,818 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:17,818 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:17,818 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 11:21:17,818 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:17,819 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 11:21:17,819 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 11:21:17,819 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 11:21:17,819 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:21:17,819 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 11:21:17,852 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:17,854 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1078318649] 11:21:17,854 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 11:21:17,854 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563] 11:21:17,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:18,45 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 11:21:18,45 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 11:21:18,45 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 11:21:18,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:18,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 11:21:18,55 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 11:21:18,55 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 11:21:18,355 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:21:18,355 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,355 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:21:18,355 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,355 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 11:21:18,355 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 11:21:18,391 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:18,422 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:18,587 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:18,587 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,588 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,589 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,601 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,656 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:21:18,656 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:21:18,656 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,656 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,656 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 11:21:18,656 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 11:21:18,656 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,656 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=null 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=null 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 11:21:18,669 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:18,669 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 11:21:18,669 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:21:18,669 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 11:21:18,669 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:21:18,669 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 11:21:18,669 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:21:18,669 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 11:21:18,669 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:21:18,669 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:18,670 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:18,670 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 11:21:18,670 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 11:21:18,674 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 11:21:18,674 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,674 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 11:21:18,674 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 11:21:18,675 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 11:21:18,679 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:21:18,679 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:18,679 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:21:18,679 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:18,743 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:18,760 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1078318649] 11:21:18,760 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 11:21:18,760 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964] 11:21:18,761 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionChainWithMultipleShards commit rateLimit adjusted to 161.33973510304187 11:21:18,761 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,761 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1715249497] 11:21:18,762 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:18,762 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:18,762 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1715249497] 11:21:18,762 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 11:21:18,762 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:18,762 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:21:18,762 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 11:21:18,762 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 11:21:18,762 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:18,762 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 11:21:18,762 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 11:21:18,762 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:18,762 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 11:21:18,762 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:18,762 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 11:21:18,762 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 11:21:18,763 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5efaead4 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@28166a41 11:21:18,768 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,768 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,769 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 11:21:18,769 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,769 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 11:21:18,769 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 11:21:18,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,772 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 11:21:18,772 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,772 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 11:21:18,772 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:21:18,772 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,772 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,774 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 11:21:18,774 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:18,774 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] resolved 11:21:18,774 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 11:21:18,774 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:18,774 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] resolved 11:21:18,774 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: successfully resolved all cohort actors 11:21:18,775 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finishCanCommit 11:21:18,775 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,775 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 1 11:21:18,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,776 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,776 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 11:21:18,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 11:21:18,776 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=10354578} is already disabled 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=10528553} is already disabled 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:18,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 11:21:18,779 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:18,779 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:18,779 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,779 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,779 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 11:21:18,779 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:18,779 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 11:21:18,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:18,781 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,781 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:18,781 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 11:21:18,781 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:18,783 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:18,784 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:18,784 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:18,788 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:18,788 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit returning result: true 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:18,788 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:18,789 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 commit 11:21:18,789 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finish commit 11:21:18,789 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,789 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 11:21:18,793 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:18,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:18,794 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:18,794 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:18,794 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}]]] 11:21:18,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 11:21:18,794 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 11:21:18,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 11:21:18,795 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 11:21:18,796 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 11:21:18,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}]]] 11:21:18,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:18,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:18,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=176} 11:21:18,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 11:21:18,798 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:21:18,798 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=180} 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:21:18,799 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,799 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 11:21:18,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%231177035563$e] 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#687213059] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,800 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#687213059] 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 11:21:18,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%23-1392575964$f] 11:21:18,800 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,800 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-551589594] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 11:21:18,801 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 11:21:18,801 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:18,801 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 11:21:18,801 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 238.7 μs 11:21:18,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-551589594] 11:21:18,801 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#687213059] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:18,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,801 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-551589594] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:18,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 92.48 μs 11:21:18,803 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit succeeded 11:21:18,804 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,804 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:18,804 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:18,804 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 11:21:18,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 11:21:18,804 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 11:21:18,804 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 11:21:18,804 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:18,804 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 11:21:18,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 11:21:18,804 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 11:21:18,804 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 11:21:18,806 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 11:21:18,806 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:18,806 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 11:21:18,806 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:21:18,806 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 11:21:18,806 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 11:21:18,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:18,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 11:21:18,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:21:18,806 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 11:21:18,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-2092095001] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 11:21:18,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-2092095001] 11:21:18,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 95.99 μs 11:21:18,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-2145556780] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-2092095001] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-2145556780] 11:21:18,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 43.02 μs 11:21:18,807 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-2145556780] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:18,807 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,807 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:18,807 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,807 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@64b55d83 11:21:18,807 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:18,808 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@659ac157 11:21:18,808 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#327432237] 11:21:18,808 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#327432237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:18,808 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#944105852] 11:21:18,810 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#327432237, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 11:21:18,811 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:21:18,811 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:21:18,811 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 11:21:18,811 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#327432237)] 11:21:18,811 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#944105852, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 11:21:18,811 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#327432237)] 11:21:18,811 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 11:21:18,818 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 11:21:18,818 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 11:21:18,818 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 11:21:18,818 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 11:21:18,818 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#944105852)] 11:21:18,818 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#944105852)] 11:21:18,824 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 11:21:18,824 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 11:21:18,824 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying 2 components for commit 11:21:18,824 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 11:21:18,824 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 11:21:18,824 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 11:21:18,824 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 11:21:18,824 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#327432237)] 11:21:18,824 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard people 11:21:18,824 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 11:21:18,824 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 11:21:18,824 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 11:21:18,824 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#944105852)] 11:21:18,825 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,825 AM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,825 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 11:21:18,825 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying 1 components for commit 11:21:18,825 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying transaction for shard people 11:21:18,825 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 11:21:18,825 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 canCommit 11:21:18,827 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,828 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,828 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,828 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,828 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 11:21:18,828 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 11:21:18,828 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,828 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,828 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 11:21:18,828 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 11:21:18,829 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:21:18,829 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 11:21:18,830 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 11:21:18,830 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 11:21:18,830 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 11:21:18,830 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 11:21:18,830 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] resolved 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] resolved 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - previous Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 readied - proceeding to FindPrimaryShard 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: successfully resolved all cohort actors 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finishCanCommit 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 directCommit called 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 sending 1 batched modifications, ready: true 11:21:18,831 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 11:21:18,833 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,833 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,833 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 11:21:18,833 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:18,833 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 11:21:18,833 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:18,833 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 11:21:18,834 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,834 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 of 1 operations, client version 13 11:21:18,834 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,834 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 ready 11:21:18,834 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 11:21:18,834 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 is scheduled for canCommit step 11:21:18,834 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:18,834 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,834 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,836 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,836 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,836 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 11:21:18,836 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:18,836 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 11:21:18,838 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:18,838 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit returning result: true 11:21:18,838 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 commit 11:21:18,838 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finish commit 11:21:18,838 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,838 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563)] 11:21:18,838 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,838 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1392575964)] 11:21:18,839 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,839 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}]]] 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,840 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 224 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 validated 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: success 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 candidate ready 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - messages: [] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 prepared 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 scheduled for commit step 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 221 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 11:21:18,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:21:18,842 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 11:21:18,842 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 submitted to persistence 11:21:18,844 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 11:21:18,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 11:21:18,844 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 11:21:18,851 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}]]] 11:21:18,851 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:18,851 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:21:18,851 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=235} 11:21:18,851 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 11:21:18,851 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 11:21:18,851 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=224} 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=221} 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 11:21:18,851 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:18,852 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 11:21:18,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 11:21:18,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 11:21:18,852 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%23-1392575964$s] 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%231177035563$r] 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 3 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 11:21:18,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards$p] 11:21:18,853 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:21:18,854 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,854 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,854 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 11:21:18,854 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 53.29 μs 11:21:18,854 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 122.0 μs 11:21:18,854 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,854 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 27.58 μs 11:21:18,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 11:21:18,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:21:18,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 11:21:18,857 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit succeeded 11:21:18,857 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 11:21:18,857 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 canCommit 11:21:18,857 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 successfully completed direct commit 11:21:18,857 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 11:21:18,858 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:18,858 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:18,858 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 11:21:18,858 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 11:21:18,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 11:21:18,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 11:21:18,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 11:21:18,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:21:18,858 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:21:18,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 11:21:18,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 11:21:18,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 11:21:18,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 11:21:18,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:21:18,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 11:21:18,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:21:18,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 11:21:18,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 11:21:18,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 11:21:18,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 3 11:21:18,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:21:18,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 11:21:18,859 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 11:21:18,859 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 11:21:18,860 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 159.4 μs 11:21:18,860 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,860 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 88.88 μs 11:21:18,860 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:21:18,860 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 53.59 μs 11:21:18,862 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 11:21:18,862 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1523999706] 11:21:18,862 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:18,862 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:18,864 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1523999706, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 11:21:18,864 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:18,864 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1523999706)] 11:21:18,867 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 11:21:18,867 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 11:21:18,867 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:18,867 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 11:21:18,867 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 11:21:18,867 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:21:18,869 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 11:21:18,869 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1487153482] 11:21:18,870 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1487153482, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 11:21:18,870 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 11:21:18,870 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1487153482)] 11:21:18,873 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 11:21:18,873 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:18,873 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:18,873 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 11:21:18,873 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 11:21:18,873 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 11:21:18,873 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 11:21:18,874 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 11:21:18,874 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 11:21:18,876 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:18,876 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:18,876 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:18,876 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:18,876 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:18,876 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:18,877 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60087f32 11:21:18,877 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60087f32 11:21:18,877 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 11:21:18,878 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:21:18,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@20600a01 11:21:18,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@20600a01 11:21:18,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 11:21:18,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:21:18,881 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 11:21:18,881 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 11:21:18,881 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:18,882 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:18,882 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:18,882 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:18,882 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:18,882 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:18,884 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 3, lastIndex: 3, isVoting: true 11:21:18,884 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:21:18,884 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:21:18,885 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 11:21:18,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 4, lastIndex: 4, isVoting: true 11:21:18,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:21:18,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:21:18,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 11:21:19,285 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:21:19,285 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1177035563]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:19,286 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 11:21:19,286 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-414: Closing 11:21:19,286 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 11:21:19,286 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-415: Closing 11:21:19,286 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:21:19,286 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:21:19,286 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 11:21:19,286 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:21:19,286 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:21:19,287 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:21:19,287 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 11:21:19,287 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:21:19,288 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:19,288 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:19,298 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:19,298 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:19,298 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:19,298 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:19,300 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:19,316 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:21:19,316 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 11:21:19,316 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 11:21:19,316 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 11:21:19,316 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 11:21:19,316 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:21:19,316 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:21:19,317 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 11:21:19,318 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:19,318 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:20,302 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3549068605584841628] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:20,332 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [740101089859104711] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:20,336 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:20,336 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:20,337 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:20.338] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:20,343 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:20,344 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:20,346 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:20.347] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:20,373 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:20,380 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5744860354975889019] 11:21:20,380 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:20,383 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:20,383 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:20,383 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:20,383 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:20,397 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:20,397 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:20,397 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:20,414 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:20,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3454381622495708669] 11:21:20,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:20,429 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:20,429 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:20,455 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:20,455 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:20,464 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:20,476 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5980664077446246712] 11:21:20,476 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:20,484 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:20,484 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:20,484 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:20,484 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:20,491 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:20,493 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:20,495 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 11:21:20,507 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 11:21:20,511 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 11:21:20,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 11:21:20,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:20,545 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:20,576 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:20,584 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 11:21:20,584 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 11:21:20,584 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@430af3f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@627d27d2 11:21:20,584 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@247fbddd 11:21:20,584 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1709032880584, None)) 11:21:20,584 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032880583,0,0)) 11:21:20,584 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@627d27d2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3e3c58fa 11:21:20,584 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7b2e4b1a 11:21:20,584 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 11:21:20,585 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:20,587 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:20,588 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 11:21:20,590 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:20,590 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 11:21:20,595 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:20,610 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:20,612 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:20,618 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:20,618 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:20,618 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#1282622837] 11:21:20,618 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 11:21:20,619 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 11:21:20,619 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#1282622837] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,620 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-1641651473 created and ready for shard:member-1-shard-cars-testInstallSnapshot 11:21:20,620 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-1642199800] 11:21:20,620 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-1641651473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,620 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-1642199800] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#1300647410] 11:21:20,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 11:21:20,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 11:21:20,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 11:21:20,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 11:21:20,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 11:21:20,621 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#2025156035] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 11:21:20,621 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 591.0 μs 11:21:20,621 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 790.9 μs: snapshotIndex=5, snapshotTerm=1, journal-size=0 11:21:20,621 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#2025156035] 11:21:20,621 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 11:21:20,621 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 82.95 μs 11:21:20,622 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#2025156035] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,622 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:20,622 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 1.693 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 11:21:20,622 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 11:21:20,622 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 11:21:20,622 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 11:21:20,622 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 11:21:20,622 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 11:21:20,622 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 11:21:20,626 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 11:21:20,636 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 11:21:20,643 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#424942186] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,677 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 11:21:20,677 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 11:21:20,677 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1487153482)] 11:21:20,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 11:21:20,677 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 11:21:20,677 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#-1523999706)] 11:21:20,677 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:20,677 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:20,677 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 11:21:20,677 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 11:21:20,677 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@60c96bca to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@78d2b5b3 11:21:20,678 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7ced2e71 11:21:20,678 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1709032880678, None)) 11:21:20,678 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032880677,0,0)) 11:21:20,678 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@78d2b5b3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@81c86c6 11:21:20,678 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 11:21:20,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@61e2d93c 11:21:20,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:20,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 11:21:20,680 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:20,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:21:20,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-2-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#-1936148085] 11:21:20,680 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 11:21:20,681 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 11:21:20,681 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#-1936148085] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,681 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#186552779] 11:21:20,681 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1709490766 created and ready for shard:member-2-shard-cars-testInstallSnapshot 11:21:20,682 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1709490766] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,682 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#186552779] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:20,682 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:20,682 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:20,685 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 11:21:20,685 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 11:21:20,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 11:21:20,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 11:21:20,686 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:20,686 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 11:21:20,686 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 11:21:20,686 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 11:21:20,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] 11:21:20,729 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 11:21:20,729 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#1282622837] 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 11:21:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 11:21:20,878 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 11:21:20,878 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 11:21:21,158 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 11:21:21,158 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 11:21:21,158 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 11:21:21,227 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:21,398 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:21,398 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:21,399 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:21,399 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 11:21:21,399 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:21,399 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:21,399 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:21:21,399 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:21,399 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:21:21,410 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:21,410 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 11:21:21,410 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:21:21,411 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:21,411 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:21:21,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:21:21,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 11:21:21,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 11:21:21,438 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 11:21:21,438 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=null 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 11:21:21,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 11:21:21,442 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:21,442 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:21,443 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 11:21:21,443 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 11:21:21,446 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 11:21:21,446 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:21:21,446 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 11:21:21,446 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 11:21:21,446 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:21,446 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:21,446 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 11:21:21,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 11:21:21,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:21:21,447 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 11:21:21,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 11:21:21,447 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 11:21:21,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 3 11:21:21,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:21:21,449 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 11:21:21,449 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@45f559e3, electionTerm=4, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 11:21:21,449 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 11:21:21,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 11:21:21,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 11:21:21,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 11:21:21,451 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 11:21:21,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 3 11:21:21,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 11:21:21,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 11:21:21,452 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 11:21:21,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 11:21:21,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=4, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 11:21:21,476 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 11:21:21,476 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 11:21:21,477 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@46f8c445, electionTerm=4, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 11:21:21,477 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 11:21:21,478 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 11:21:21,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 11:21:21,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 11:21:21,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 11:21:21,478 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 11:21:21,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:21:21,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1782487953] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 11:21:21,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 355.3 μs 11:21:21,479 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 11:21:21,480 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1782487953] 11:21:21,480 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 11:21:21,480 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 141.8 μs 11:21:21,480 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1782487953] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:21,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=4, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 11:21:21,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 11:21:21,483 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 11:21:21,483 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 11:21:21,484 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 11:21:21,484 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] 11:21:21,484 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 Queuing TransactionOperation 11:21:21,484 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:21,484 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#1282622837, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@9e87c1a] 11:21:21,485 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#1282622837, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@9e87c1a] 11:21:21,485 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#1282622837)] for shard cars 11:21:21,485 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:21:21,485 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3d3cfca7 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=4, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 11:21:21,487 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7920d0bf 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7920d0bf 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 11:21:21,487 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:21:21,488 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 11:21:21,488 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:21,488 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:21,490 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 11:21:21,491 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 11:21:21,491 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:21,491 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:21,492 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testInstallSnapshot, matchIndex: 5, lastIndex: 5, isVoting: true 11:21:21,492 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Follower's log matches - sending ElectionTimeout 11:21:21,492 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:21:21,492 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leader transfer complete 11:21:21,494 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:21,897 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 11:21:21,897 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] ShardManager - shard-manager-testInstallSnapshot: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#1282622837]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:21,897 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 11:21:21,897 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-420: Closing 11:21:21,898 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 11:21:21,898 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 11:21:21,898 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 11:21:21,898 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 11:21:21,898 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: shutting down 11:21:21,899 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 11:21:21,899 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 11:21:21,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:21,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:21,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:21,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#823192659] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:21,899 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1465503834] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1465503834] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:21,900 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:21,900 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:21,908 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:21,908 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:21,908 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:21,909 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:21,912 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:21,929 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 11:21:21,929 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 11:21:21,929 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 11:21:21,930 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 11:21:21,930 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 11:21:21,930 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:21,931 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:22,924 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5744860354975889019] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:22,949 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:22,949 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:22,951 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:22.953] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:22,957 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3454381622495708669] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:22,965 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:22,965 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:22,967 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:22.968] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:22,986 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:22,995 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6236450342928927816] 11:21:22,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:22,997 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:22,997 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:22,998 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:22,998 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:23,11 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:23,11 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:23,11 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:23,16 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:23,23 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1278435266850685061] 11:21:23,23 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:23,25 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:23,25 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:23,30 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:23,30 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:23,50 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:23,68 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3832665236253422051] 11:21:23,68 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:23,69 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:23,69 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:23,70 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:23,70 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:23,82 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:23,84 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:23,86 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 11:21:23,96 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:23,105 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:23,112 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,118 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 11:21:23,118 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:21:23,118 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@132bfb6d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@480fa2f4 11:21:23,118 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@65e5a2ea 11:21:23,118 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709032883118, None)) 11:21:23,118 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:23,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,119 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:23,119 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,119 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032883117,0,0)) 11:21:23,119 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@480fa2f4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@17043edc 11:21:23,119 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5fbb5a38 11:21:23,119 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 11:21:23,120 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:23,122 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:23,123 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:23,125 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 11:21:23,126 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:23,126 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,142 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,142 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,142 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] 11:21:23,142 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 11:21:23,143 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 11:21:23,148 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,148 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1194439389] 11:21:23,148 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1931679862 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,149 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1931679862] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,149 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1194439389] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,151 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:23,159 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:23,159 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:23,159 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 11:21:23,159 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 11:21:23,159 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:23,159 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 11:21:23,159 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 11:21:23,160 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 11:21:23,160 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 11:21:23,164 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1707533457] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,179 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 11:21:23,179 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:21:23,179 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:23,180 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6f71742c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@181fdfb9 11:21:23,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,180 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@563dc0d0 11:21:23,180 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:23,180 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,180 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709032883180, None)) 11:21:23,181 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032883179,0,0)) 11:21:23,181 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@181fdfb9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4785619f 11:21:23,181 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 11:21:23,182 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@13931e85 11:21:23,182 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:23,182 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,184 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,184 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,184 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1070285413] 11:21:23,184 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 11:21:23,184 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1070285413] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,185 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#719031746 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,185 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 11:21:23,186 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#719031746] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,187 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:23,187 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:23,187 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 11:21:23,187 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:23,187 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 11:21:23,187 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 11:21:23,187 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,187 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 11:21:23,187 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 11:21:23,188 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 11:21:23,202 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-325491329] 11:21:23,202 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-325491329] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:23,212 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:23,219 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:23,232 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] 11:21:23,232 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 11:21:23,232 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] 11:21:23,398 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:23,398 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 11:21:23,398 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 11:21:23,398 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,399 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 11:21:23,399 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:23,399 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 11:21:23,399 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:23,399 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 11:21:23,399 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 11:21:23,399 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 11:21:23,678 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 11:21:23,678 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,678 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 11:21:23,793 AM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:23,967 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 11:21:23,967 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:23,967 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 11:21:24,19 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:24,19 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,19 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:24,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,37 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:24,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,37 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,37 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,82 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:24,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 11:21:24,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,288 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 11:21:24,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,291 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:24,291 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 11:21:24,291 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:24,291 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:24,291 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 11:21:24,291 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:24,291 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:24,293 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:24,294 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:24,294 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:24,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 11:21:24,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:24,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:24,295 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:24,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:21:24,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 11:21:24,295 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 11:21:24,296 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 11:21:24,296 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:24,296 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:24,296 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 11:21:24,296 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 11:21:24,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:24,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:24,296 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 11:21:24,296 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 11:21:24,296 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:24,296 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:24,297 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:24,297 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 11:21:24,297 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 11:21:24,297 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:24,341 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:24,343 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:24,346 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 11:21:24,350 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1560095455] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:24,352 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:24,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 11:21:24,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:21:24,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@cd1528c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@30519c48 11:21:24,355 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@14e14973 11:21:24,355 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709032884354, None)) 11:21:24,355 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032884353,0,0)) 11:21:24,355 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@30519c48 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@507e6307 11:21:24,355 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 11:21:24,356 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:24,359 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:24,360 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:24,360 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:24,366 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:24,366 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:24,366 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:24,366 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:24,368 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:24,383 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] 11:21:24,383 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 11:21:24,383 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] 11:21:24,383 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] 11:21:24,417 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 11:21:25,37 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:21:25,373 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-3832665236253422051] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:25,377 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-3832665236253422051] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:30,142 AM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 11:21:30,144 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:30,144 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:30,159 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:30,164 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 11:21:30,327 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1707533457] 11:21:30,327 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 11:21:30,327 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1070285413] 11:21:30,327 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] 11:21:30,327 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:21:30,328 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1070285413] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,328 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:21:30,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:21:30,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:21:30,329 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,329 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1070285413] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,329 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985] 11:21:30,332 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:21:30,332 AM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:21:30,333 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:30,333 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:21:30,333 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:30,333 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 11:21:30,333 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 11:21:30,333 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 11:21:30,334 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12861bd7 11:21:30,334 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12861bd7 11:21:30,334 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 11:21:30,334 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 11:21:30,334 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:30,334 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:30,334 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:30,334 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:30,336 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 11:21:30,336 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 11:21:30,336 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:30,336 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:30,336 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 11:21:30,336 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 11:21:30,336 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:30,337 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:30,337 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 11:21:30,747 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 11:21:30,748 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#1254375985]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:30,748 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:30,748 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-425: Closing 11:21:30,748 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 11:21:30,748 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:30,748 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:30,748 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:21:30,748 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,749 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:30,749 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 11:21:30,749 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:30,749 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,749 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,749 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#39851412] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:30,757 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:30,757 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:31,223 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:31,223 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 11:21:31,770 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:31,771 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:31,773 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6236450342928927816] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [02/27/2024 11:21:31.779] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:31,800 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:21:31,800 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:21:31,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 11:21:31,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 11:21:31,801 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:21:31,803 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:31,803 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:31,806 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:31.809] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:31,833 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:31,842 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2261045588657809676] 11:21:31,842 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:31,844 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:31,844 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:31,846 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:31,846 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:31,864 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:31,870 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:31,870 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:31,870 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:31,873 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-642547991885893304] 11:21:31,874 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:31,876 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:31,876 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:31,877 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:31,877 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:31,899 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:31,906 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7578779155260987790] 11:21:31,906 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:31,909 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:31,909 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:31,909 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:31,909 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:31,920 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:31,922 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:31,923 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 11:21:31,934 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:21:31,937 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,938 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 11:21:31,938 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:21:31,938 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@a47dc41 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ed8a1c4 11:21:31,938 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@ae734c7 11:21:31,938 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709032891938, None)) 11:21:31,939 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032891937,0,0)) 11:21:31,940 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ed8a1c4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@522ed450 11:21:31,940 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 11:21:31,940 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:31,941 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:31,943 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:31,946 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 11:21:31,947 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 11:21:31,948 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,948 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:31,948 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,948 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@431eb37f 11:21:31,948 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:31,948 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,950 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:31,952 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,952 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,952 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018] 11:21:31,952 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 11:21:31,952 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,953 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 11:21:31,953 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1099331729 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,953 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#377645143] 11:21:31,953 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1099331729] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,954 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#377645143] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,955 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:31,955 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:31,955 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:31,955 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 11:21:31,955 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 11:21:31,955 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 11:21:31,955 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 11:21:31,956 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 11:21:31,957 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 11:21:31,957 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1852185957] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,958 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:21:31,958 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 11:21:31,958 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:21:31,959 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4f19ce76 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1cb6344e 11:21:31,959 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7a1060ae 11:21:31,959 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709032891959, None)) 11:21:31,959 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032891958,0,0)) 11:21:31,960 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1cb6344e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7b27b5f 11:21:31,960 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 11:21:31,962 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 11:21:31,962 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,962 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:31,962 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,962 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5d4081b8 11:21:31,962 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:31,962 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,963 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1098874755] 11:21:31,963 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 11:21:31,964 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 11:21:31,964 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1098874755] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,964 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#-1907872283] 11:21:31,964 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1691180910 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,964 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#-1907872283] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,968 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:31,968 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:31,970 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:31,979 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 11:21:31,979 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 11:21:31,980 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1691180910] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:31,983 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:31,985 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:31,985 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 11:21:31,985 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:31,985 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 11:21:31,985 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 11:21:31,985 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 11:21:32,10 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] 11:21:32,11 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 11:21:32,11 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018] 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 11:21:32,199 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 11:21:32,199 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 11:21:32,499 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:32,518 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:21:32,518 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,518 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 11:21:32,828 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:21:32,828 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,828 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 11:21:32,860 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:32,860 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:32,861 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:32,861 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,861 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:32,861 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,861 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,870 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:32,870 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,870 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,870 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:32,870 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,870 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:32,921 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:33,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:21:33,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,118 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 11:21:33,119 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,121 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:33,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 11:21:33,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:33,122 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:33,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 11:21:33,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:33,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 11:21:33,124 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 11:21:33,124 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 11:21:33,125 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:33,125 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:33,125 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 11:21:33,125 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 11:21:33,127 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 11:21:33,127 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,128 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 11:21:33,128 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 11:21:33,128 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:33,128 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:33,128 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 11:21:33,129 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 11:21:33,129 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:33,217 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1852185957] 11:21:33,218 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 11:21:33,218 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1098874755] 11:21:33,218 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,218 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1852185957] 11:21:33,218 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 11:21:33,218 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1098874755] 11:21:33,230 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,230 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,230 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-197-0 of 1 operations, client version 13 11:21:33,233 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-197-0 ready 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 validated 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-197-0: success 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 candidate ready 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: canCommit - candidate rootPath: / 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: canCommit - messages: [] 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: preCommit - successfulFromPrevious: [] 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 prepared 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-197-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:33,234 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:33,235 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,235 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 submitted to persistence 11:21:33,235 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:21:33,235 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:21:33,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:21:33,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:21:33,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:33,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:21:33,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:21:33,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 11:21:33,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:33,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:21:33,237 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 11:21:33,239 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,239 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=7000021} is already disabled 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 11:21:33,240 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:33,241 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 11:21:33,242 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:21:33,242 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:21:33,242 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:21:33,242 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:21:33,242 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:21:33,242 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: commit - successfulFromPrevious: [] 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from COMMIT_PENDING to COMMITTED 11:21:33,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-761#-960909201] 11:21:33,244 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#172349352] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 11:21:33,244 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:21:33,245 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:33,245 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:21:33,245 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#172349352] 11:21:33,245 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:21:33,246 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#172349352] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:33,246 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 100.6 μs 11:21:33,249 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 11:21:33,249 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:33,249 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 11:21:33,249 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:21:33,249 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 11:21:33,249 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-197-0 11:21:33,250 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 11:21:33,250 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] 11:21:33,250 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-153972312] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 11:21:33,250 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 11:21:33,250 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:21:33,250 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-153972312] 11:21:33,251 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:33,251 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@459f03bd] 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-153972312] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@459f03bd] 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 110.5 μs 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018)] for shard cars 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:21:33,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@22d69fb3 11:21:33,251 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,251 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,252 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 11:21:33,255 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,255 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-198-0 of 1 operations, client version 13 11:21:33,255 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,255 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-198-0 ready 11:21:33,256 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 canCommit 11:21:33,256 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018)] resolved 11:21:33,256 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: successfully resolved all cohort actors 11:21:33,257 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 finishCanCommit 11:21:33,257 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-198-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018)] 11:21:33,257 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018)] 11:21:33,257 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,257 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,257 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 validated 11:21:33,257 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:33,257 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-198-0: success 11:21:33,257 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:33,257 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit returning result: true 11:21:33,257 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 commit 11:21:33,257 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 finish commit 11:21:33,257 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-198-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018)] 11:21:33,257 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018)] 11:21:33,257 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,257 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 candidate ready 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit - candidate rootPath: / 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit - messages: [] 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: preCommit - successfulFromPrevious: [] 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 prepared 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-198-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 11:21:33,258 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 submitted to persistence 11:21:33,259 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:21:33,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 11:21:33,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:21:33,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:33,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 11:21:33,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:21:33,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 11:21:33,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:33,261 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: commit - successfulFromPrevious: [] 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from COMMIT_PENDING to COMMITTED 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%231144973018$a] 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:33,262 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:21:33,263 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: commit succeeded 11:21:33,263 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:21:33,263 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 107.1 μs 11:21:33,263 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 11:21:33,263 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:33,263 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@22d69fb3 11:21:33,263 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:33,264 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:21:33,264 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,264 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 11:21:33,264 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 11:21:33,264 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 11:21:33,264 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:21:33,264 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:33,264 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@49a72924 11:21:33,264 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:33,264 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@49a72924 11:21:33,264 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 11:21:33,264 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:33,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 11:21:33,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:33,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 11:21:33,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:21:33,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 11:21:33,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-198-0 11:21:33,267 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:21:33,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:21:33,268 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:21:33,268 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 11:21:33,268 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 149.0 μs 11:21:33,268 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:21:33,268 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:33,268 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:33,269 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 11:21:33,269 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 11:21:33,269 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:33,269 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 11:21:33,482 AM [cluster-test-shard-dispatcher-14] [WARN] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 210, lastApplied : 2, commitIndex : 2 11:21:33,679 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 11:21:33,679 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1144973018]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:33,679 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-431: Closing 11:21:33,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:21:33,680 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:21:33,680 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:21:33,680 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:21:33,680 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:21:33,681 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:33,681 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 11:21:33,681 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:33,681 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:21:33,681 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:33,681 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#2133798994] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:33,691 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:33,691 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:33,698 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:33,698 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:33,698 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:33,698 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:33,702 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:33,723 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:21:33,723 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:21:33,723 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 11:21:33,723 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:21:33,723 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:21:33,725 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:33,725 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:34,710 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2261045588657809676] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:34,742 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-642547991885893304] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:34,742 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:34,742 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:34,747 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:34.749] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:34,761 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:34,761 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:21:34.767] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:34,790 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:34,797 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6727522208260283189] 11:21:34,797 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:34,799 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:34,799 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:34,800 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:34,800 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:34,813 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:34,816 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:34,816 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:34,816 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:34,829 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5093508988960021465] 11:21:34,830 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:34,831 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:34,831 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:34,837 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:34,837 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:34,853 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:34,866 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8100022876517964063] 11:21:34,868 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:34,869 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:34,869 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:34,880 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:34,880 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:34,882 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:34,883 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:34,885 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 11:21:34,907 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:34,908 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1046407017] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,908 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:21:34,910 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:21:34,910 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:21:34,910 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@39252fef to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1413732 11:21:34,910 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 11:21:34,910 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,910 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:34,910 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,912 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:34,915 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6c7d93e 11:21:34,915 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709032894910, None)) 11:21:34,916 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032894909,0,0)) 11:21:34,916 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3c35e362 11:21:34,916 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1413732 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@574ef7d8 11:21:34,916 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 11:21:34,916 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:34,918 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:34,920 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 11:21:34,920 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:34,920 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,921 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-859718861] 11:21:34,921 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 11:21:34,921 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 11:21:34,925 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-859718861] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,930 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-54835698 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,930 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1728139111] 11:21:34,932 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-54835698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,932 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1728139111] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,933 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:21:34,935 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#305094616] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,937 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 11:21:34,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,937 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:21:34,937 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:21:34,937 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4d417101 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@238498da 11:21:34,938 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@668612ce 11:21:34,938 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:34,938 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,938 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709032894938, None)) 11:21:34,938 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032894937,0,0)) 11:21:34,938 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@238498da to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@54395a60 11:21:34,938 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 11:21:34,938 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@c0100e7 11:21:34,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:34,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,939 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-2051016734] 11:21:34,939 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 11:21:34,940 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 11:21:34,940 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-2051016734] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,940 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-920255479 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,940 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-920255479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,941 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:34,941 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:34,941 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 11:21:34,941 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 11:21:34,941 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:34,941 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 11:21:34,941 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 11:21:34,941 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 11:21:34,941 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:34,941 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 11:21:34,941 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:34,942 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 11:21:34,942 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 11:21:34,944 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1727062263] 11:21:34,944 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1727062263] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:34,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:34,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 11:21:34,944 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 11:21:34,944 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:34,945 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 11:21:34,945 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 11:21:34,946 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:34,949 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:34,989 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1046407017] 11:21:34,989 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 11:21:34,989 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-859718861] 11:21:35,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:35,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:21:35,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 11:21:35,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,206 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 11:21:35,206 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:35,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 11:21:35,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:35,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 11:21:35,206 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:21:35,207 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 11:21:35,435 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:35,435 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,435 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 11:21:35,500 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:35,704 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:35,704 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,704 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 11:21:35,817 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:35,817 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:35,817 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:35,817 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,817 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,817 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:35,817 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,817 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,818 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,832 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:35,832 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,832 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,832 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:35,832 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,832 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:35,896 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:36,5 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:36,5 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,5 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 11:21:36,5 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,8 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:36,10 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:36,10 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 11:21:36,11 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 11:21:36,11 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 11:21:36,13 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 11:21:36,13 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,13 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:21:36,13 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:36,13 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:36,13 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 11:21:36,13 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 11:21:36,14 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:21:36,14 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:36,45 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithSingleShard commit rateLimit adjusted to 161.33973510304187 11:21:36,46 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,46 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#305094616] 11:21:36,46 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:36,46 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:36,46 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:36,46 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:36,46 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:36,46 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 11:21:36,47 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 11:21:36,47 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars 11:21:36,47 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 11:21:36,47 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 11:21:36,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 11:21:36,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 11:21:36,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 11:21:36,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:36,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:36,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 11:21:36,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,50 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028] 11:21:36,50 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:36,50 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 11:21:36,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 11:21:36,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:21:36,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:36,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 11:21:36,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 11:21:36,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 11:21:36,52 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, version=13] 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 3 batched modifications, ready: false 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028)] 11:21:36,52 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028)] 11:21:36,53 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 11:21:36,53 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 11:21:36,53 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=5271591} is already disabled 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:36,54 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:21:36,57 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:21:36,60 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=3] 11:21:36,60 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 11:21:36,60 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 11:21:36,60 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 11:21:36,60 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 1 batched modifications, ready: false 11:21:36,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028)] 11:21:36,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028)] 11:21:36,65 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=1] 11:21:36,65 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 DataExists operation succeeded 11:21:36,65 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 11:21:36,65 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 11:21:36,65 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 11:21:36,65 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 0 batched modifications, ready: true 11:21:36,65 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1366181028)] 11:21:36,65 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 11:21:36,67 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, client version 13 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: canCommit for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: success 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 249 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 11:21:36,67 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:36,68 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:36,68 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,68 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 11:21:36,68 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 11:21:36,68 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 11:21:36,68 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 11:21:36,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:21:36,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:36,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=249} 11:21:36,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 11:21:36,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 11:21:36,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:36,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 11:21:36,69 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 11:21:36,69 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:36,69 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 11:21:36,69 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:21:36,69 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 11:21:36,69 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,70 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:36,70 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:36,70 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithSingleShard_member-1-shard-cars-testReadWriteTransactionWithSingleShard_shard-cars-member-2%3Adatastore-testReadWriteTransactionWithSingleShard%400%3A0_8%23-1366181028$f] 11:21:36,70 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-470643226] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 11:21:36,70 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:21:36,70 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:36,70 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 11:21:36,71 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-470643226] 11:21:36,71 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:21:36,71 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-470643226] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:36,71 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 105.1 μs 11:21:36,72 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 11:21:36,72 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 11:21:36,72 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 11:21:36,72 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:36,72 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:36,72 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 11:21:36,72 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 11:21:36,72 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 11:21:36,72 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 11:21:36,72 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:36,72 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 11:21:36,72 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:21:36,72 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 11:21:36,72 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:36,73 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-347685754] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 11:21:36,73 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:21:36,73 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 11:21:36,73 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-347685754] 11:21:36,73 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:21:36,74 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 51.83 μs 11:21:36,74 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-347685754] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:36,74 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 11:21:36,74 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1039590340] 11:21:36,74 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1039590340] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:36,78 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1039590340, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 11:21:36,78 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:36,78 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1039590340)] 11:21:36,84 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 11:21:36,84 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:21:36,84 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,85 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c78da13 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c78da13 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:36,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:36,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 11:21:36,86 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:21:36,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:36,86 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:36,87 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 11:21:36,87 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 11:21:36,87 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:36,87 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 11:21:36,495 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 11:21:36,495 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-859718861]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:36,495 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,495 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-436: Closing 11:21:36,495 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:21:36,496 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:21:36,496 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:21:36,496 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:21:36,497 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:21:36,497 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 11:21:36,497 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:21:36,498 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:36,498 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:36,504 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:36,504 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:36,504 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:36,504 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:36,506 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:36,516 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:21:36,517 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:21:36,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 11:21:36,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:21:36,517 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:21:36,524 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:36,525 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:37,516 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6727522208260283189] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:37,542 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:37,543 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:37,545 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:37.546] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:37,554 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5093508988960021465] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:37,561 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:37,561 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:37,564 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:37.566] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:37,583 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:37,590 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8642891702836838205] 11:21:37,590 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:37,593 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:37,593 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:37,603 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:37,603 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:37,604 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:37,604 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:37,604 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:37,609 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:37,617 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8983548428653534544] 11:21:37,617 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:37,619 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:37,619 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:37,629 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:37,629 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:37,635 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:37,651 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6578041444610593736] 11:21:37,651 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:37,656 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:37,656 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:37,656 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:37,656 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:37,662 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:37,663 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:37,665 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 11:21:37,673 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,674 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 11:21:37,681 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 11:21:37,681 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:37,681 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@485d0f5e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74221686 11:21:37,681 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@701d8cda 11:21:37,681 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709032897681, None)) 11:21:37,681 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032897680,0,0)) 11:21:37,681 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74221686 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4e0c9b1 11:21:37,681 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 11:21:37,682 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:37,683 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 11:21:37,683 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,684 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@45247dae 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,684 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893] 11:21:37,685 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 11:21:37,685 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 11:21:37,685 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 11:21:37,686 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 11:21:37,686 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#2139124925 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,686 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-10146403] 11:21:37,686 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 11:21:37,686 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,686 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,686 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#2139124925] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,686 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-10146403] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,687 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1265249882] 11:21:37,687 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#123344561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,690 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#123344561 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,692 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1265249882] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,693 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#955595002] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,694 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 11:21:37,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:37,696 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:37,696 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 11:21:37,696 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 11:21:37,696 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:37,696 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:37,697 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:21:37,697 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 11:21:37,697 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 11:21:37,697 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 11:21:37,697 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 11:21:37,697 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:37,698 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:21:37,701 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:37,709 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 11:21:37,710 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 11:21:37,710 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:37,710 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7b3c9355 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9113abd 11:21:37,710 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@78bd412e 11:21:37,710 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 11:21:37,710 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:37,710 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 11:21:37,710 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:37,710 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,710 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,710 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709032897710, None)) 11:21:37,711 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032897709,0,0)) 11:21:37,712 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9113abd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@45bec33e 11:21:37,712 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5be3b68a 11:21:37,712 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 11:21:37,712 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:37,712 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:37,713 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:37,714 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,714 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:37,714 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:37,714 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 11:21:37,714 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 11:21:37,714 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,714 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-1623811836 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:37,714 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-538822794] 11:21:37,714 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-1623811836] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,715 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-538822794] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,716 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,716 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:21:37,716 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-1547143541] 11:21:37,716 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 11:21:37,717 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#1959547827 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 11:21:37,717 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1172806516] 11:21:37,717 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 11:21:37,717 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:37,719 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-1547143541] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,719 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#1959547827] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,719 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1172806516] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:37,720 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:37,720 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:37,720 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:21:37,720 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 11:21:37,720 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,720 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 11:21:37,720 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 11:21:37,720 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 11:21:37,720 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 11:21:37,721 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:21:37,721 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 11:21:37,721 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 11:21:37,721 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:21:37,721 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 11:21:37,722 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:21:37,722 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 11:21:37,728 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:37,728 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:37,728 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:21:37,728 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 11:21:37,728 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:37,728 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 11:21:37,728 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 11:21:37,728 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 11:21:37,728 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:21:37,729 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 11:21:37,762 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] 11:21:37,763 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:21:37,763 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 11:21:37,950 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 11:21:37,950 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 11:21:38,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:38,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 11:21:38,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 11:21:38,19 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,20 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 11:21:38,20 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:38,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 11:21:38,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:21:38,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 11:21:38,20 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 11:21:38,20 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 11:21:38,209 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:21:38,209 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:21:38,210 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 11:21:38,265 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:38,269 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:21:38,269 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,269 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 11:21:38,325 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:38,479 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:21:38,479 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:21:38,479 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 11:21:38,520 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:21:38,520 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,520 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 11:21:38,620 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:38,620 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,621 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,631 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,674 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:38,709 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:21:38,709 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:21:38,709 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 11:21:38,709 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:21:38,713 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:38,715 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:38,715 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:38,715 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 11:21:38,715 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:38,715 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:38,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 11:21:38,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 11:21:38,716 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 11:21:38,716 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:21:38,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:21:38,716 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 11:21:38,717 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:21:38,718 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 11:21:38,718 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:21:38,718 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 11:21:38,719 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 11:21:38,719 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:38,719 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:21:38,719 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:21:38,719 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:21:38,809 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:21:38,809 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,809 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 11:21:38,810 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 11:21:38,813 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 11:21:38,813 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 11:21:38,814 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 11:21:38,814 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:38,815 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 11:21:38,815 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:21:38,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:38,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:38,815 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 11:21:38,815 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 11:21:38,815 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:38,831 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 11:21:38,878 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] 11:21:38,878 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 11:21:38,878 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893] 11:21:38,879 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSnapshotOnRootOverwrite commit rateLimit adjusted to 155.8689248333021 11:21:38,879 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,881 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] 11:21:38,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:38,881 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] 11:21:38,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3066b917] 11:21:38,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:21:38,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@608da74b] 11:21:38,881 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3066b917] 11:21:38,881 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars 11:21:38,881 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:21:38,881 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:38,882 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@608da74b] 11:21:38,882 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] for shard default 11:21:38,882 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard default resolved to local data tree - adding local factory 11:21:38,882 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3eaf0e5f 11:21:38,882 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying 2 components for commit 11:21:38,882 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard cars 11:21:38,882 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 11:21:38,882 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:38,883 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard default 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,883 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 11:21:38,883 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,883 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 canCommit 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,883 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 11:21:38,883 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 11:21:38,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] resolved 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] resolved 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: successfully resolved all cohort actors 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finishCanCommit 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:38,884 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:38,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 11:21:38,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:38,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 11:21:38,884 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,884 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,888 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:38,888 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:38,888 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:38,888 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,888 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,888 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 11:21:38,888 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:38,888 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 11:21:38,888 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:38,889 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:38,889 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit returning result: true 11:21:38,889 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 commit 11:21:38,889 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finish commit 11:21:38,889 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:38,889 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 11:21:38,889 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:38,889 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 11:21:38,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,889 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 282 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:38,890 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 11:21:38,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=7780819} is already disabled 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:38,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=8056445} is already disabled 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 11:21:38,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=282} 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:38,892 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 11:21:38,892 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:21:38,893 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 11:21:38,893 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:38,893 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 11:21:38,893 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 11:21:38,893 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:21:38,893 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:38,893 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:38,893 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$a] 11:21:38,894 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1605269225] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 11:21:38,894 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:38,894 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 11:21:38,894 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:38,894 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7152a15c, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%23-902644893$b] 11:21:38,895 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit succeeded 11:21:38,895 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1377970894] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 11:21:38,895 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 11:21:38,895 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 11:21:38,896 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:38,896 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 11:21:38,896 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:38,896 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:38,896 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:38,896 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:38,896 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] for shard default, trying to use DirectTransactionContextWrapper 11:21:38,896 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3eaf0e5f 11:21:38,896 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:38,896 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1605269225] 11:21:38,896 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1377970894] 11:21:38,896 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:38,896 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:38,896 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 11:21:38,896 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:38,897 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:38,897 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:21:38,897 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 94.30 μs 11:21:38,897 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 11:21:38,897 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:38,897 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 11:21:38,897 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 86.30 μs 11:21:38,897 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:38,897 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1605269225] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:38,897 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1377970894] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:38,897 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 11:21:38,897 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:38,897 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 11:21:38,897 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:21:38,898 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 11:21:38,898 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,898 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#677150326] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 11:21:38,899 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:38,899 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 11:21:38,899 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:21:38,899 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 11:21:38,899 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:38,899 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 11:21:38,899 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:21:38,899 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#677150326] 11:21:38,899 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-89793108] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 11:21:38,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] 11:21:38,899 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 109.7 μs 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:21:38,899 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 11:21:38,900 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:38,900 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#677150326] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:38,900 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 11:21:38,900 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-89793108] 11:21:38,900 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:38,900 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:38,900 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:38,900 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 34.89 μs 11:21:38,900 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@44fa7502, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:38,900 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 11:21:38,900 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 11:21:38,900 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-89793108] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:38,900 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:38,900 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5d542636, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:38,900 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 11:21:38,900 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 11:21:38,901 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 11:21:38,901 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 11:21:38,901 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 11:21:38,901 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 11:21:38,901 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:38,901 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:38,901 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 11:21:38,901 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:38,901 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:38,901 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 11:21:38,902 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:38,902 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2a3b44a, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:38,902 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 11:21:38,902 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 11:21:38,902 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 11:21:38,902 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:21:38,902 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:38,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:38,903 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 11:21:38,903 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 11:21:38,903 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:38,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#955595002] 11:21:38,903 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:21:38,903 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 11:21:38,903 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:38,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:38,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:39,109 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,109 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,109 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,109 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,109 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying 1 components for commit 11:21:39,110 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying transaction for shard cars 11:21:39,110 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 = [ 1 ] 11:21:39,110 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,110 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 canCommit 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 validated 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: success 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 candidate ready 11:21:39,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - candidate rootPath: / 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - messages: [] 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 prepared 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 11:21:39,111 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 submitted to persistence 11:21:39,112 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 11:21:39,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 11:21:39,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:39,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 11:21:39,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:21:39,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: commit - successfulFromPrevious: [] 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,115 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$c] 11:21:39,116 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:39,116 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 11:21:39,116 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 successfully completed direct commit 11:21:39,116 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed successfully 11:21:39,116 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,116 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 113.6 μs 11:21:39,117 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 11:21:39,117 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,117 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,117 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,117 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,117 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,117 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,118 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,118 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying 1 components for commit 11:21:39,118 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying transaction for shard cars 11:21:39,118 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 = [ 1 ] 11:21:39,118 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,118 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 canCommit 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 validated 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: success 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 11:21:39,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:39,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 candidate ready 11:21:39,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - candidate rootPath: / 11:21:39,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - messages: [] 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 prepared 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 11:21:39,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:21:39,119 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,119 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 11:21:39,119 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 submitted to persistence 11:21:39,119 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 11:21:39,120 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,120 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 303.6 μs 11:21:39,120 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 11:21:39,121 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 11:21:39,121 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,121 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,121 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:21:39,121 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,121 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 11:21:39,121 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,121 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,122 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 3, nextIndex: 4 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 3 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: commit - successfulFromPrevious: [] 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,122 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$d] 11:21:39,123 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 11:21:39,123 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:21:39,123 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 11:21:39,123 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 successfully completed direct commit 11:21:39,123 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed successfully 11:21:39,123 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,123 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 46.86 μs 11:21:39,123 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 11:21:39,123 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,123 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,123 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,123 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,123 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,123 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,123 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,123 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying 1 components for commit 11:21:39,123 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying transaction for shard cars 11:21:39,123 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 = [ 1 ] 11:21:39,123 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,124 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 canCommit 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 3 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 validated 11:21:39,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 3 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: success 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 candidate ready 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - candidate rootPath: / 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - messages: [] 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: preCommit - successfulFromPrevious: [] 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 prepared 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,124 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,124 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 11:21:39,124 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,125 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 submitted to persistence 11:21:39,125 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 11:21:39,125 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 11:21:39,125 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 104.6 μs 11:21:39,125 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 11:21:39,126 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 11:21:39,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:21:39,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 11:21:39,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,127 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 4, nextIndex: 5 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 4 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: commit - successfulFromPrevious: [] 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$e] 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:21:39,128 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 11:21:39,128 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 successfully completed direct commit 11:21:39,128 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed successfully 11:21:39,128 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,128 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 11:21:39,128 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 50.21 μs 11:21:39,128 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,128 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,129 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,129 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,129 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,129 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,129 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,129 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying 1 components for commit 11:21:39,129 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying transaction for shard cars 11:21:39,129 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 = [ 1 ] 11:21:39,129 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,129 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 canCommit 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 validated 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: success 11:21:39,129 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 4 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,129 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,129 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 11:21:39,129 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 4 11:21:39,129 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 11:21:39,130 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 candidate ready 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - candidate rootPath: / 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - messages: [] 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: preCommit - successfulFromPrevious: [] 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 prepared 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,130 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 11:21:39,130 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 11:21:39,130 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 11:21:39,130 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 submitted to persistence 11:21:39,130 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 67.24 μs 11:21:39,130 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 11:21:39,133 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,133 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,133 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 5, nextIndex: 6 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 5 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: commit - successfulFromPrevious: [] 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$f] 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:21:39,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 11:21:39,134 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 successfully completed direct commit 11:21:39,134 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,134 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed successfully 11:21:39,134 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 11:21:39,134 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,134 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,135 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,135 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 72.66 μs 11:21:39,135 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,135 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,135 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,135 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,135 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying 1 components for commit 11:21:39,135 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying transaction for shard cars 11:21:39,135 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 = [ 1 ] 11:21:39,135 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,135 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 canCommit 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 5 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 5 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 validated 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: success 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 candidate ready 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - candidate rootPath: / 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - messages: [] 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: preCommit - successfulFromPrevious: [] 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 prepared 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,136 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 11:21:39,136 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 11:21:39,136 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 228.6 μs 11:21:39,136 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,137 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 11:21:39,137 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,137 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 11:21:39,137 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 submitted to persistence 11:21:39,137 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,137 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,138 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 6, nextIndex: 7 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 6 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: commit - successfulFromPrevious: [] 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$g] 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:21:39,139 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 11:21:39,139 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 successfully completed direct commit 11:21:39,139 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed successfully 11:21:39,139 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,139 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 11:21:39,139 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,139 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,139 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 64.40 μs 11:21:39,139 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,140 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,140 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,140 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,140 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,140 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying 1 components for commit 11:21:39,140 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying transaction for shard cars 11:21:39,140 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 = [ 1 ] 11:21:39,140 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,140 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 canCommit 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,141 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 6 11:21:39,141 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,141 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 6 11:21:39,141 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,141 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 validated 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: success 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,141 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 candidate ready 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - candidate rootPath: / 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - messages: [] 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: preCommit - successfulFromPrevious: [] 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 prepared 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,141 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,141 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,142 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 11:21:39,142 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,142 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 88.40 μs 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 submitted to persistence 11:21:39,142 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 11:21:39,143 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 11:21:39,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 11:21:39,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 11:21:39,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,143 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,144 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 7, nextIndex: 8 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 7 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: commit - successfulFromPrevious: [] 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$h] 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:21:39,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 11:21:39,145 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 successfully completed direct commit 11:21:39,145 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed successfully 11:21:39,145 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,145 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 11:21:39,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,145 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 60.49 μs 11:21:39,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,145 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,145 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying 1 components for commit 11:21:39,145 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying transaction for shard cars 11:21:39,145 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 = [ 1 ] 11:21:39,145 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,145 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 canCommit 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 validated 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: success 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 candidate ready 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - candidate rootPath: / 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - messages: [] 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: preCommit - successfulFromPrevious: [] 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 prepared 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 11:21:39,146 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 submitted to persistence 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 7 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 7 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 11:21:39,147 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 11:21:39,147 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 72.00 μs 11:21:39,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 11:21:39,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=8, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,148 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 11:21:39,149 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 8, nextIndex: 9 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 8 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: commit - successfulFromPrevious: [] 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$i] 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 11:21:39,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 11:21:39,154 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 successfully completed direct commit 11:21:39,154 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed successfully 11:21:39,154 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,154 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 11:21:39,154 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 78.53 μs 11:21:39,154 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,154 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,154 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,154 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,154 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,154 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,154 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,154 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying 1 components for commit 11:21:39,154 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying transaction for shard cars 11:21:39,154 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 = [ 1 ] 11:21:39,154 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,154 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 canCommit 11:21:39,154 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,154 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,154 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 validated 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: success 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 candidate ready 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - candidate rootPath: / 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - messages: [] 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: preCommit - successfulFromPrevious: [] 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 prepared 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 9, leaderSnapShotIndex: 7 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 11:21:39,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 submitted to persistence 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 8 11:21:39,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 8 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 8 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 11:21:39,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 11:21:39,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 11:21:39,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 11:21:39,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 11:21:39,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=9, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 9, nextIndex: 10 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 9, lastAppliedIndex: 8 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 9 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: commit - successfulFromPrevious: [] 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$j] 11:21:39,159 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 11:21:39,159 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 9, leaderSnapShotIndex: 8 11:21:39,159 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 11:21:39,159 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 successfully completed direct commit 11:21:39,159 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed successfully 11:21:39,159 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,159 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 68.03 μs 11:21:39,159 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 11:21:39,159 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,160 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,160 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying 1 components for commit 11:21:39,160 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying transaction for shard cars 11:21:39,160 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 = [ 1 ] 11:21:39,160 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,160 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 canCommit 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 validated 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: success 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 candidate ready 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - candidate rootPath: / 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - messages: [] 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: preCommit - successfulFromPrevious: [] 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 prepared 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,163 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,163 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 119.7 μs 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 10, leaderSnapShotIndex: 8 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 11:21:39,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 submitted to persistence 11:21:39,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 11:21:39,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 11:21:39,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 9 11:21:39,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 9, context.getLastApplied(): 8, lastIndex(): 9 11:21:39,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 9 11:21:39,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 11:21:39,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 11:21:39,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 11:21:39,166 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,166 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 74.74 μs 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 11:21:39,166 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 9, entries to be added from: 0 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,167 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 11:21:39,167 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 10, nextIndex: 11 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 9 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 10 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: commit - successfulFromPrevious: [] 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,167 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$k] 11:21:39,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 11:21:39,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 11:21:39,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 11:21:39,168 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 successfully completed direct commit 11:21:39,168 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed successfully 11:21:39,168 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,168 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 11:21:39,168 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,168 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,168 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,168 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 72.09 μs 11:21:39,168 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,168 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,168 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,168 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,168 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying 1 components for commit 11:21:39,168 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying transaction for shard cars 11:21:39,168 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 = [ 1 ] 11:21:39,168 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,169 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 canCommit 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 validated 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: success 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 candidate ready 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - candidate rootPath: / 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - messages: [] 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: preCommit - successfulFromPrevious: [] 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 prepared 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 11, leaderSnapShotIndex: 9 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 11:21:39,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 submitted to persistence 11:21:39,171 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 10 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 9, lastIndex(): 10 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 10 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 11:21:39,173 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,173 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=11, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,173 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 99.27 μs 11:21:39,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,174 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 11:21:39,174 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 11, nextIndex: 12 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 11 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: commit - successfulFromPrevious: [] 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$l] 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 11:21:39,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 11:21:39,175 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 successfully completed direct commit 11:21:39,175 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed successfully 11:21:39,175 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,175 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 11:21:39,175 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,175 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,175 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,175 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 77.70 μs 11:21:39,176 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] 11:21:39,176 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:21:39,176 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 11 11:21:39,177 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 11 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 11:21:39,177 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 11:21:39,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 11:21:39,177 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 65.39 μs 11:21:39,178 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#955595002] 11:21:39,178 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:21:39,178 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:39,178 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:39,381 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,381 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,381 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,381 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,381 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] for shard default, trying to use DirectTransactionContextWrapper 11:21:39,381 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3eaf0e5f 11:21:39,381 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,382 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying 2 components for commit 11:21:39,382 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard cars 11:21:39,382 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 11:21:39,382 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,382 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard default 11:21:39,382 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,382 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 11:21:39,382 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:39,382 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,382 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 canCommit 11:21:39,382 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,382 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 11:21:39,382 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,382 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] resolved 11:21:39,382 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,382 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 11:21:39,382 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] resolved 11:21:39,382 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: successfully resolved all cohort actors 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finishCanCommit 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:39,383 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:39,383 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit returning result: true 11:21:39,384 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 commit 11:21:39,384 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finish commit 11:21:39,384 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,384 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] 11:21:39,384 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:39,384 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] 11:21:39,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 11:21:39,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 11:21:39,384 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,384 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,384 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 11:21:39,384 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 11:21:39,384 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 11:21:39,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 114 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 12, leaderSnapShotIndex: 10 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 12 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:39,385 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 11:21:39,385 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 11:21:39,386 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 11:21:39,386 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 11:21:39,387 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 11:21:39,387 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,387 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:39,387 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=114} 11:21:39,387 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 11:21:39,387 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:21:39,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 11:21:39,387 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:21:39,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 11, entries to be added from: 0 11:21:39,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 11:21:39,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 11:21:39,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 11:21:39,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:39,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 11:21:39,388 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 12, nextIndex: 13 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 12, lastAppliedIndex: 11 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 12 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%23-902644893$m] 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 11:21:39,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23728525933$n] 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:39,389 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:39,390 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 11:21:39,390 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:39,390 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit succeeded 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 11:21:39,390 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:39,390 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 9 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 11 11:21:39,390 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,390 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 11:21:39,390 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 29 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:39,390 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 85.56 μs 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@18979341, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:39,390 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 11:21:39,390 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 11:21:39,390 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@40b466fb, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 11:21:39,391 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 11:21:39,391 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@79b9e276 11:21:39,391 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,391 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-902644893)] for shard default, trying to use DirectTransactionContextWrapper 11:21:39,391 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3eaf0e5f 11:21:39,391 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 9 11:21:39,391 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 29 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 9 11:21:39,391 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 29 11:21:39,391 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:39,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:39,391 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 11:21:39,391 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 1.432 ms 11:21:39,393 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] 11:21:39,393 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:21:39,393 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:39,393 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933] 11:21:39,391 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 11:21:39,398 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#955595002] 11:21:39,399 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:21:39,399 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:39,399 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:39,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 11:21:39,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:39,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 11:21:39,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 11:21:39,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 11:21:39,399 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 12 11:21:39,399 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 12, context.getLastApplied(): 11, lastIndex(): 12 11:21:39,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,399 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 12 11:21:39,399 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 11:21:39,399 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 11:21:39,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:39,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 11:21:39,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 11:21:39,400 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:39,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 11:21:39,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:39,400 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 11:21:39,400 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 81.48 μs 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 11:21:39,400 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 11:21:39,400 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 14.92 μs 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 27 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4d18bee8, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:39,400 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 11:21:39,401 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@777e1e4d, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 27 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 27 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 11:21:39,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 11:21:39,403 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:39,403 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 11:21:39,453 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2135709139] 11:21:39,640 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:21:39,656 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:39,656 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3eab5752 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3eab5752 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@590b8711 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@590b8711 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 11:21:39,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 11:21:39,658 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:21:39,658 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:39,658 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:39,658 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:21:39,658 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:21:39,658 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:21:39,662 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 11:21:39,662 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 11:21:39,662 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:21:39,662 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:39,662 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:39,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 12, lastIndex: 12, isVoting: true 11:21:39,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 11:21:39,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 11:21:39,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 11:21:39,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 2, lastIndex: 2, isVoting: true 11:21:39,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 11:21:39,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 11:21:39,664 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 11:21:39,673 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:21:39,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:21:39,680 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:21:40,69 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 11:21:40,69 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#728525933]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:40,70 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 11:21:40,70 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 11:21:40,70 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-444: Closing 11:21:40,70 AM [cluster-test-shard-dispatcher-16] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-443: Closing 11:21:40,70 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:21:40,70 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:21:40,70 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 11:21:40,71 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:21:40,71 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:21:40,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 11:21:40,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:21:40,71 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:21:40,71 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#165493425] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:40,72 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:40,72 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:40,79 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:40,79 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:40,79 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:40,79 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:40,82 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:40,91 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:21:40,91 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 11:21:40,91 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 11:21:40,92 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:21:40,93 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 11:21:40,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 11:21:40,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:21:40,93 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 11:21:40,93 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:40,93 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:41,93 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8642891702836838205] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:41,107 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:41,108 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:41,111 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:41.112] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:41,113 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8983548428653534544] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:41,128 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:41,128 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:41,130 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:41.132] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:41,151 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:41,157 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1764411377263255290] 11:21:41,157 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:41,158 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:41,158 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:41,160 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:41,160 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:41,169 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:41,169 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:41,169 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:41,172 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:41,179 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3986652559441405737] 11:21:41,179 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:41,181 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:41,181 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:41,186 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:41,186 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:41,198 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:41,210 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3507282221865258614] 11:21:41,210 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:41,212 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:41,212 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:41,212 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:41,212 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:41,222 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:41,223 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:41,224 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 11:21:41,250 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 11:21:41,255 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1917979451] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,273 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 11:21:41,273 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,273 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:41,273 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,274 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:21:41,274 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:21:41,274 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@a9d6aed to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@129bf2ac 11:21:41,274 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@16a46427 11:21:41,274 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709032901274, None)) 11:21:41,275 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032901273,0,0)) 11:21:41,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1f2dac8d 11:21:41,275 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@129bf2ac to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@75e22b17 11:21:41,276 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 11:21:41,276 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:41,278 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:41,279 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 11:21:41,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:41,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,291 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:41,295 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684] 11:21:41,295 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 11:21:41,295 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 11:21:41,295 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:41,295 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,296 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:41,296 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 11:21:41,296 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#344517094 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,296 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#344517094] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,296 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#922542054] 11:21:41,296 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#922542054] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,299 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:41,299 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:41,299 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:41,299 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:21:41,299 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 11:21:41,299 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 11:21:41,299 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 11:21:41,299 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 11:21:41,301 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:41,301 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2107856575] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,313 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 11:21:41,316 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 11:21:41,316 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:41,316 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:41,316 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,317 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:21:41,317 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:21:41,317 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4e42826e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20ce741d 11:21:41,317 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@65ccfe60 11:21:41,326 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709032901323, None)) 11:21:41,326 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032901322,0,0)) 11:21:41,327 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20ce741d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7ded04a 11:21:41,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@26eb83c1 11:21:41,327 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 11:21:41,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:41,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:41,328 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,328 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:41,328 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#27142616] 11:21:41,328 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 11:21:41,329 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 11:21:41,329 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#27142616] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,330 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#890320352 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:41,330 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#890320352] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,331 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:41,331 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:41,331 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:41,331 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:21:41,331 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,331 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1864586210] 11:21:41,331 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 11:21:41,331 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 11:21:41,331 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 11:21:41,332 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1864586210] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:41,332 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 11:21:41,332 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 11:21:41,377 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1917979451] 11:21:41,377 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 11:21:41,378 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684] 11:21:41,565 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 11:21:41,565 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#1039590340)] 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:41,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:21:41,588 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:21:41,588 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 11:21:41,837 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:41,837 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:41,837 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 11:21:41,921 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:42,117 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:42,117 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:42,117 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 11:21:42,179 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:42,179 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 11:21:42,179 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:42,179 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:42,179 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,179 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:42,179 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,179 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:42,179 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:42,186 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:42,186 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 11:21:42,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,186 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:42,186 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,225 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:42,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:42,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:42,377 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 11:21:42,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:42,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 11:21:42,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:42,379 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:42,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 11:21:42,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:42,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:42,380 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:42,381 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:42,381 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:42,381 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 11:21:42,381 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:42,381 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:42,382 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:21:42,382 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 11:21:42,382 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:21:42,382 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:21:42,382 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:42,382 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:42,382 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 11:21:42,382 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 11:21:42,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 11:21:42,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:42,383 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 11:21:42,384 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithSingleShard commit rateLimit adjusted to 155.8689248333021 11:21:42,384 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,384 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2107856575] 11:21:42,384 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:42,384 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:42,385 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:42,385 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:42,385 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 11:21:42,385 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 11:21:42,385 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:42,385 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 11:21:42,385 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:21:42,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:42,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:42,385 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 11:21:42,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:42,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 11:21:42,385 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 4 batched modifications, ready: true 11:21:42,386 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 11:21:42,387 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:21:42,388 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:42,388 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 4 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 of 4 operations, client version 13 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: success 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 245 11:21:42,389 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 11:21:42,390 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 11:21:42,390 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:42,390 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:42,390 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,390 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 11:21:42,390 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 11:21:42,390 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 11:21:42,391 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=3204524} is already disabled 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 11:21:42,392 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=245} 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 11:21:42,393 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 11:21:42,394 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$a] 11:21:42,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#2054571034] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 11:21:42,395 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#2054571034] 11:21:42,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:21:42,395 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:21:42,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:42,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 11:21:42,395 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#2054571034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:42,395 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 127.3 μs 11:21:42,396 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 11:21:42,396 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 11:21:42,396 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 11:21:42,397 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:42,397 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:42,397 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 11:21:42,397 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 11:21:42,397 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 11:21:42,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 1 11:21:42,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:42,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 11:21:42,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:21:42,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 11:21:42,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,398 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1101712176] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 11:21:42,398 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1101712176] 11:21:42,398 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:21:42,398 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 11:21:42,398 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:21:42,398 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1101712176] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:42,398 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 127.4 μs 11:21:42,403 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 11:21:42,404 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#1622313767] 11:21:42,404 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#1622313767, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 11:21:42,404 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:42,404 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#1622313767)] 11:21:42,404 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#1622313767] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:42,408 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 11:21:42,408 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:42,408 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1917979451] 11:21:42,408 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:21:42,408 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:42,408 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@647377ac] 11:21:42,409 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@647377ac] 11:21:42,409 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684)] for shard cars 11:21:42,409 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:21:42,409 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5752ab71 11:21:42,409 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:42,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:42,409 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 11:21:42,409 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:21:42,409 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying 1 components for commit 11:21:42,409 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying transaction for shard cars 11:21:42,409 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 directCommit called 11:21:42,409 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 sending 1 batched modifications, ready: true 11:21:42,409 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 11:21:42,409 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 canCommit 11:21:42,410 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,410 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 of 1 operations, client version 13 11:21:42,410 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,410 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 ready 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 validated 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: success 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 candidate ready 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - candidate rootPath: / 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - messages: [] 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 prepared 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 212 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,411 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 11:21:42,412 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 submitted to persistence 11:21:42,412 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=212} 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:42,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: commit - successfulFromPrevious: [] 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 11:21:42,413 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$d] 11:21:42,414 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:21:42,414 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:21:42,414 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:42,414 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 11:21:42,414 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 98.10 μs 11:21:42,415 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 modifications completed with CommitTransactionReply [version=13] 11:21:42,415 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 successfully completed direct commit 11:21:42,415 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 11:21:42,416 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:42,416 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:42,416 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 11:21:42,416 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 11:21:42,416 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Queuing TransactionOperation 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:42,416 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:21:42,416 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 11:21:42,416 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 52.75 μs 11:21:42,417 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 11:21:42,418 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#377058369, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0, version=13] 11:21:42,418 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 11:21:42,418 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#377058369)] 11:21:42,422 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#377058369] 11:21:42,422 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#377058369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:42,423 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 DataExists operation succeeded 11:21:42,423 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 11:21:42,423 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:42,423 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:42,423 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 11:21:42,423 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 11:21:42,423 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Queuing TransactionOperation 11:21:42,424 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 11:21:42,424 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1663025263] 11:21:42,424 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1663025263] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:42,425 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1663025263, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0, version=13] 11:21:42,425 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:42,425 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1663025263)] 11:21:42,427 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 ReadData operation succeeded 11:21:42,427 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 11:21:42,427 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:42,427 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5752ab71 11:21:42,427 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 11:21:42,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1917979451] 11:21:42,427 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 11:21:42,428 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684] 11:21:42,428 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1524509684] 11:21:42,479 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:21:42,479 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:42,479 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-449: Closing 11:21:42,479 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:21:42,479 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 11:21:42,480 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:42,480 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:42,490 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:42,491 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:42,491 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:42,491 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:42,493 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:42,499 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:21:42,499 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:42,499 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 11:21:42,499 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:21:42,499 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 11:21:42,500 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:42,500 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:43,505 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1764411377263255290] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:43,513 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:43,514 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:43,515 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3986652559441405737] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:43,515 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:43.516] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:43,548 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:21:43,559 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [207917456662554794] 11:21:43,559 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:43,562 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:43,562 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 11:21:43,562 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:43,562 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:43,572 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:43,574 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:43,576 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 11:21:43,587 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 11:21:43,590 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1089371836] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,592 AM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:21:43,592 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 11:21:43,592 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:43,592 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 11:21:43,592 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@717267a6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fbcf56d 11:21:43,592 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@b836042 11:21:43,592 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 11:21:43,593 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709032903592, None)) 11:21:43,593 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032903591,0,0)) 11:21:43,593 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fbcf56d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7f05d730 11:21:43,593 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@48108ea7 11:21:43,593 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 11:21:43,593 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:43,593 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:43,594 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 11:21:43,594 AM [reinstated-member2-shard-dispatcher-16] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 11:21:43,594 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 11:21:43,595 AM [reinstated-member2-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 11:21:43,595 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 11:21:43,595 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#130929364 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:43,595 AM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#983613837] 11:21:43,595 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-378468260] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,595 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#2071475199] 11:21:43,595 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#-141969238] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,596 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#130929364] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,596 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#983613837] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,596 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#2071475199] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,596 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1369075235 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 11:21:43,596 AM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1369075235] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 11:21:43,596 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 11:21:43,596 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 62 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 245 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 11:21:43,596 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 1 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 11:21:43,597 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@701abb0f}, purged=MutableUnsignedLongSet{size=0}} 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 212 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 2 to 2 11:21:43,597 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 1.491 ms - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 11:21:43,598 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 11:21:43,599 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 11:21:43,599 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:21:43,599 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 11:21:43,599 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 11:21:43,601 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:21:43,601 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:21:43,601 AM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:21:43,601 AM [reinstated-member2-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 11:21:43,601 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:43,601 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:43,601 AM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 11:21:43,601 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:21:43,601 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 11:21:43,601 AM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:21:43,601 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:43,601 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:43,601 AM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 11:21:43,601 AM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 11:21:43,643 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1089371836] 11:21:43,644 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 11:21:43,644 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-378468260] 11:21:43,644 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 11:21:43,644 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1089371836] 11:21:43,644 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:43,644 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 Queuing TransactionOperation 11:21:43,644 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-378468260, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2ae6f421] 11:21:43,644 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-378468260, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2ae6f421] 11:21:43,644 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0: Found primary ActorSelection[Anchor(akka://reinstated-member2/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-378468260)] for shard cars 11:21:43,644 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:21:43,645 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@2ae05a16 11:21:43,645 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 11:21:43,645 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:43,645 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 11:21:43,645 AM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 11:21:43,645 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 11:21:43,645 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@307481db 11:21:43,645 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@307481db 11:21:43,646 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 11:21:43,646 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1a03d657 11:21:43,646 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1a03d657 11:21:43,646 AM [reinstated-member2-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:21:43,646 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-454: Closing 11:21:43,646 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 11:21:43,646 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-453: Closing 11:21:43,646 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:21:43,646 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:21:43,646 AM [reinstated-member2-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 11:21:43,646 AM [reinstated-member2-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 11:21:43,646 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:21:43,647 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:21:43,647 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:21:43,647 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:21:43,647 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:21:43,647 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:21:43,648 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:21:43,648 AM [reinstated-member2-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 11:21:43,648 AM [reinstated-member2-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:21:43,648 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1089371836] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1089371836] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,649 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1089371836] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1089371836] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,649 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://reinstated-member2/user/$a#1534663075] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#1534663075] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,649 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:43,649 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:43,654 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:43,666 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:43,666 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:43,667 AM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:43.668] [reinstated-member2-akka.actor.internal-dispatcher-5] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:43,678 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:43,684 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5175443478403786786] 11:21:43,684 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:43,685 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:43,685 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:43,686 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:43,686 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:43,695 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:43,695 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:43,695 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:43,700 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:43,706 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4765616528969312246] 11:21:43,706 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:43,713 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:43,713 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:43,713 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:43,713 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:43,727 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:43,742 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5385458232372082494] 11:21:43,742 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:43,744 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:43,744 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:43,749 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:43,749 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:43,753 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:43,757 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:43,758 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,768 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-741780254] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,770 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,773 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:43,774 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 11:21:43,774 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:21:43,774 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@913829c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c5731c 11:21:43,774 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1eb7ad3f 11:21:43,775 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709032903775, None)) 11:21:43,775 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,775 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,775 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:43,775 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,776 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032903774,0,0)) 11:21:43,776 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2bec2e0f 11:21:43,776 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c5731c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@76216bdd 11:21:43,776 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:43,776 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 11:21:43,776 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:43,778 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:43,781 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,781 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:43,781 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,781 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,781 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,781 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#495989536] 11:21:43,782 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 11:21:43,782 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 11:21:43,782 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#495989536] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,782 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1580615191 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,783 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1580615191] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,783 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#541087525] 11:21:43,783 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#541087525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,787 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:43,787 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:43,787 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:43,787 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:21:43,788 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 11:21:43,788 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 11:21:43,788 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 11:21:43,788 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,788 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 11:21:43,788 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,793 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:43,795 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:43,798 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-477195536] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,801 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 11:21:43,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:21:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5a6d1cb2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e90a58 11:21:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@57d7f58 11:21:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709032903801, None)) 11:21:43,801 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:43,801 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032903800,0,0)) 11:21:43,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4cbf9d63 11:21:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e90a58 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@467e8e36 11:21:43,801 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 11:21:43,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:43,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,802 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1780482705] 11:21:43,802 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 11:21:43,803 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1780482705] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,803 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#561087420] 11:21:43,803 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1552379066 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,803 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1552379066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,803 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 11:21:43,803 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#561087420] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:43,803 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:43,803 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:43,804 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:43,804 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:21:43,804 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 11:21:43,804 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,804 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 11:21:43,804 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 11:21:43,804 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:43,804 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 11:21:43,852 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-741780254] 11:21:43,852 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 11:21:43,852 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#495989536] 11:21:44,26 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:44,26 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 11:21:44,26 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 11:21:44,26 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,27 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 11:21:44,27 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:44,27 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 11:21:44,27 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:44,27 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:21:44,27 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 11:21:44,27 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 11:21:44,256 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:21:44,256 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,256 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 11:21:44,386 AM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:44,546 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:21:44,546 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,546 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 11:21:44,697 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:44,697 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:44,698 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:44,699 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,699 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,699 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:44,699 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,699 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,699 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,707 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:44,707 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,707 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,707 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:44,707 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,707 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,765 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:44,806 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:21:44,806 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,806 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 11:21:44,806 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 11:21:44,811 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 11:21:44,811 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 11:21:44,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:21:44,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,813 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 11:21:44,813 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 11:21:44,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:44,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:44,813 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 11:21:44,814 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 11:21:44,814 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:44,858 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 11:21:44,858 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-477195536] 11:21:44,858 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Queuing TransactionOperation 11:21:44,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:44,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 11:21:44,858 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 11:21:44,858 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars 11:21:44,859 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 11:21:44,859 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 11:21:44,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, disabling transaction tracking 11:21:44,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 11:21:44,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 11:21:44,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:44,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:44,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 11:21:44,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Creating transaction : member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 11:21:44,862 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1056756884] 11:21:44,862 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1056756884] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:44,862 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 11:21:44,863 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1056756884, transactionId=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0, version=13] 11:21:44,863 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:44,863 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1056756884)] 11:21:44,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 11:21:44,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Number of entries to be appended = 1 11:21:44,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:44,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 11:21:44,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 11:21:44,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Log size is now 1 11:21:44,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:44,864 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 11:21:44,864 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done true 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: 0, nextIndex: 1 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Setting last applied to 0 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 11:21:44,864 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=4154556} is already disabled 11:21:44,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:44,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Commit index set to 0 11:21:44,867 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 ReadData operation succeeded 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Setting last applied to 0 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 11:21:44,867 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 11:21:44,868 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:21:44,868 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,868 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-459: Closing 11:21:44,868 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 11:21:44,868 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,869 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:44,869 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:44,874 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:44,874 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:44,874 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:44,875 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:44,876 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:44,890 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 11:21:44,988 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,989 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithCreateTxFailureDueToNoLeader commit rateLimit adjusted to 155.8689248333021 11:21:44,989 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 11:21:44,989 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,989 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:44,989 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:44,989 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 11:21:44,989 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 11:21:44,989 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:21:44,989 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:44,989 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 11:21:44,989 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 11:21:44,989 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:21:44,989 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 11:21:44,990 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying 1 components for commit 11:21:44,990 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying transaction for shard cars 11:21:44,990 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 11:21:44,990 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 canCommit 11:21:45,16 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:21:45,46 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:21:45,47 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 11:21:45,47 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 11:21:45,76 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:21:45,106 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:21:45,106 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 11:21:45,106 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 11:21:45,136 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:21:45,166 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:21:45,166 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 11:21:45,166 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:21:45,177 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 11:21:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:45,178 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:45,178 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 11:21:45,196 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:21:45,226 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:21:45,226 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-477195536] 11:21:45,226 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:45,226 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 11:21:45,386 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:21:45,386 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:45,387 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 11:21:45,387 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:45,446 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 11:21:45,446 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 11:21:45,446 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Find primary for shard cars failed org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:45,447 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Creating NoOpTransaction because of error org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:45,447 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:45,447 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:45,448 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:21:45,448 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:21:45,448 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:21:45,448 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:21:45,448 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:21:45,449 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:21:45,449 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:21:45,450 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:21:45,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 11:21:45,450 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:21:45,451 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:45,451 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:45,884 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5175443478403786786] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:46,464 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4765616528969312246] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:46,470 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:46,470 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:46,471 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:46.473] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:46,488 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:46,488 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:46,495 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:46.496] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:46,515 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:46,523 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7098214593691270057] 11:21:46,523 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:46,525 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:46,525 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:46,536 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:46,536 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:46,545 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:46,545 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:46,545 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:46,561 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:46,574 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8704437924893585465] 11:21:46,575 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:46,576 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:46,576 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:46,586 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:46,586 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:46,625 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:46,627 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:46,632 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:46,634 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8039477234546094452] 11:21:46,634 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:46,641 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:46,641 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:46,645 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:46,645 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:46,645 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:46,646 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:46,648 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 11:21:46,653 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1051098420] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,653 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:21:46,680 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 11:21:46,680 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:46,680 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@25e73dd0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@163e66bb 11:21:46,680 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@533cea18 11:21:46,680 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 11:21:46,681 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,681 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,681 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:46,681 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,681 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,686 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709032906686, None)) 11:21:46,686 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032906685,0,0)) 11:21:46,686 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@68ddfc88 11:21:46,686 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@163e66bb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5e447cd4 11:21:46,687 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 11:21:46,687 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:46,687 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:46,687 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,689 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:46,690 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 11:21:46,694 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:21:46,695 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-425146775] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,697 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,697 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,697 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2004454966] 11:21:46,697 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,697 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,697 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,697 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-64187306] 11:21:46,698 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 11:21:46,698 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:21:46,698 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 11:21:46,699 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:21:46,699 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2004454966] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,699 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-64187306] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,699 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#2086030234] 11:21:46,699 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1722433843 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,699 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1722433843] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,699 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1342133951 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,699 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-737662826] 11:21:46,707 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 11:21:46,707 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:46,707 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2f473d48 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4fb89452 11:21:46,707 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@f1a15af 11:21:46,707 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 11:21:46,707 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,707 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,707 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:46,708 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,708 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,708 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709032906708, None)) 11:21:46,708 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032906707,0,0)) 11:21:46,709 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1902bbda 11:21:46,709 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4fb89452 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@346266b8 11:21:46,709 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 11:21:46,709 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:46,710 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,710 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,710 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,710 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1062132827] 11:21:46,710 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,710 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 11:21:46,710 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:21:46,711 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 11:21:46,711 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1062132827] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,711 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-905252931] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,711 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:21:46,711 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:46,711 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:46,711 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:46,711 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:46,711 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:46,711 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:21:46,711 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:46,711 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:21:46,711 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 11:21:46,711 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 11:21:46,711 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:21:46,711 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:21:46,711 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:21:46,713 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#2049936071] 11:21:46,713 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1098005452] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,713 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#2049936071] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,714 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:46,716 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#2086030234] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,716 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1342133951] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,716 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-737662826] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,716 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:46,717 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1354678207] 11:21:46,717 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1354678207] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,718 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1251084578 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,718 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1251084578] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:46,722 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:46,722 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:46,723 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:46,723 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:21:46,723 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:46,723 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:46,723 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,723 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:46,723 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:21:46,723 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:21:46,724 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 11:21:46,724 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 11:21:46,724 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,724 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-905252931] 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 11:21:46,724 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,724 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 11:21:46,724 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:21:46,725 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 11:21:46,725 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1098005452 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:46,725 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:21:46,725 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:21:46,725 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 11:21:46,760 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1051098420] 11:21:46,760 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 11:21:46,760 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2004454966] 11:21:46,998 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:46,998 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:21:46,999 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 11:21:46,999 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 11:21:47,17 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:47,17 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:21:47,18 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 11:21:47,18 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 11:21:47,252 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:47,298 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:21:47,298 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,298 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 11:21:47,332 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:47,338 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:21:47,338 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,338 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 11:21:47,549 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:47,549 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,551 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,559 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,588 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:21:47,588 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,588 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 11:21:47,588 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,590 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:47,590 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 11:21:47,590 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:47,590 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:47,590 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=null 11:21:47,591 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:47,591 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,591 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 11:21:47,593 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 11:21:47,593 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 11:21:47,593 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 11:21:47,593 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:47,593 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:21:47,593 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:21:47,593 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 11:21:47,595 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 11:21:47,595 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:21:47,595 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:47,595 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:47,595 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 11:21:47,597 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 11:21:47,597 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,598 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:21:47,598 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:47,598 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:47,598 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 11:21:47,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:21:47,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:47,618 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:21:47,618 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,618 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 11:21:47,618 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=null 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,620 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 11:21:47,622 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 11:21:47,622 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 11:21:47,623 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 11:21:47,623 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,623 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 11:21:47,623 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:21:47,623 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:47,623 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:47,623 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 11:21:47,624 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:21:47,624 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:47,655 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:47,711 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 11:21:47,715 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1051098420] 11:21:47,715 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 11:21:47,715 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-64187306] 11:21:47,716 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 1823971002, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@cdbd369} 11:21:47,717 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithMultipleShards commit rateLimit adjusted to 155.8689248333021 11:21:47,717 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:47,717 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-425146775] 11:21:47,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:21:47,717 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:47,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:21:47,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-425146775] 11:21:47,718 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] for shard people 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:21:47,718 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:47,718 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] for shard cars 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 11:21:47,718 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:47,718 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 11:21:47,718 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 11:21:47,718 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 11:21:47,718 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 11:21:47,718 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 11:21:47,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] 11:21:47,719 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 11:21:47,719 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 11:21:47,719 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 11:21:47,719 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 11:21:47,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] 11:21:47,719 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 11:21:47,719 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 11:21:47,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 11:21:47,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:47,729 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 11:21:47,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@450b9cd2 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1a8f87ae 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:47,730 AM [cluster-test-shard-dispatcher-31] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:47,733 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,734 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Readying Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 11:21:47,736 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 11:21:47,736 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 ready 11:21:47,736 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 11:21:47,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 11:21:47,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:47,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:47,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 11:21:47,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 11:21:47,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:47,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 11:21:47,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:47,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:47,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 11:21:47,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 11:21:47,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:47,743 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,743 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 11:21:47,743 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 11:21:47,743 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,743 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:47,743 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 11:21:47,744 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:47,743 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a cohort Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:47,744 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:47,744 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@20cb948b failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$4.onFailure(ThreePhaseCommitCohortProxy.java:140) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:102) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:95) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=15130168} is already disabled 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=14933194} is already disabled 11:21:47,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 11:21:47,744 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 abort 11:21:47,745 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:47,745 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:47,745 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a abort cohort path Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:47,745 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 11:21:47,745 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,745 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 11:21:47,745 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:47,747 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:47,748 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 11:21:47,748 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:47,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:47,749 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 3 11:21:47,750 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:47,755 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@20cb948b is no longer running 11:21:47,755 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:47,756 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:47,756 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:47,756 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 11:21:47,756 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 11:21:47,756 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 11:21:47,756 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 11:21:47,756 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 11:21:47,756 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:47,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6fb63a30 11:21:47,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6fb63a30 11:21:47,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 11:21:47,757 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:47,757 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7550730d 11:21:47,757 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:47,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:47,757 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:47,757 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:47,757 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:47,758 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:47,759 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:21:47,759 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:21:47,759 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:47,759 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 11:21:47,968 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 11:21:47,968 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:47,970 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:21:47,970 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:21:47,970 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:47,970 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 11:21:47,980 AM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 218, lastApplied : 1, commitIndex : 1 11:21:48,168 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:21:48,168 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2004454966]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:48,169 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:48,169 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-467: Closing 11:21:48,169 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 11:21:48,169 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:48,169 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-466: Closing 11:21:48,169 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 1 pending queued transactions 11:21:48,169 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:21:48,170 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:21:48,170 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:21:48,170 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 11:21:48,170 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:21:48,170 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:21:48,170 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#324139861] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#324139861] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:48,171 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1051098420] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1051098420] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:48,171 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#324139861] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#324139861] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:48,171 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:48,171 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:21:48.177] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-743)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:21:48.177] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-743)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:21:48.178] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-743)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:21:48.178] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-743)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:48,195 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:21:48,196 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:21:48,196 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:21:48,196 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:21:48,196 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:21:48,196 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 11:21:48,196 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 11:21:48,196 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:21:48,197 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:48,197 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:49,185 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7098214593691270057] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:49,214 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:49,215 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:49,215 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8704437924893585465] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:49,217 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:49.218] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:49,224 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:49,224 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:49,226 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:49.227] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:49,242 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:21:49,249 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1680027953749489702] 11:21:49,249 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:49,250 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:49,250 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:49,257 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:49,257 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:49,263 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:49,263 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:49,263 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:49,267 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:49,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7965598113198990047] 11:21:49,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:49,275 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:49,275 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:49,276 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:49,276 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:49,293 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:21:49,300 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3190083142434908656] 11:21:49,301 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:49,302 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:49,302 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:49,311 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:49,312 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:49,314 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 11:21:49,321 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:49,321 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:49,331 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-716593625] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,332 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 11:21:49,333 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 11:21:49,333 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:49,333 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5b53d434 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d10598 11:21:49,334 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 11:21:49,334 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,335 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,335 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:49,335 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,335 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,337 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@47712321 11:21:49,337 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709032909335, None)) 11:21:49,337 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032909334,0,0)) 11:21:49,337 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d10598 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@780f7a64 11:21:49,338 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@36b5bd74 11:21:49,338 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 11:21:49,338 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:49,340 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:49,341 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,342 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306] 11:21:49,343 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 11:21:49,343 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 11:21:49,344 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 11:21:49,344 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1182373778 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,347 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 11:21:49,353 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-906089221] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,357 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1936420814 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,358 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-1783973069] 11:21:49,358 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-263631921] 11:21:49,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:49,359 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:49,360 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 11:21:49,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:49,361 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 11:21:49,369 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 11:21:49,369 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:49,369 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@9ee3596 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5036627a 11:21:49,369 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5409eca4 11:21:49,369 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709032909369, None)) 11:21:49,369 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032909368,0,0)) 11:21:49,369 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5036627a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@43b27801 11:21:49,369 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4427f58a 11:21:49,369 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 11:21:49,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:49,370 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:49,373 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 11:21:49,374 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 11:21:49,374 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 11:21:49,374 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:49,374 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:49,376 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:49,376 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:21:49,376 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 11:21:49,377 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,377 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,377 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1182373778] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,377 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1936420814] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,377 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-1783973069] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,377 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-263631921] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,377 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:21:49,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 11:21:49,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 11:21:49,383 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 11:21:49,396 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:49,396 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:49,396 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 11:21:49,396 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:21:49,396 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 11:21:49,404 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 11:21:49,404 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 11:21:49,405 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#97780231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,405 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#2129741245 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:49,405 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#2129741245] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,405 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-684328338] 11:21:49,405 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-684328338] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,407 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 11:21:49,408 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 11:21:49,409 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-26670549 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 11:21:49,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-907742662] 11:21:49,410 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:49,410 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:49,412 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 11:21:49,412 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 11:21:49,412 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 11:21:49,413 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:49,413 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:49,413 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:49,413 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:49,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 11:21:49,413 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 11:21:49,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 11:21:49,414 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:21:49,414 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#1200820574] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,414 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-26670549] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,414 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-907742662] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:49,414 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:49,414 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:21:49,415 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 11:21:49,415 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:21:49,419 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 11:21:49,419 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 11:21:49,420 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-716593625] 11:21:49,424 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 11:21:49,424 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:49,428 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:49,431 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:49,431 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:49,431 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:49,431 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:49,431 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,431 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,433 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:49,435 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:49,678 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:49,678 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 11:21:49,679 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:21:49,679 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,679 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 11:21:49,679 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:49,679 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:21:49,679 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:49,679 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:21:49,679 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:21:49,680 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 11:21:49,688 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:49,688 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 11:21:49,688 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 11:21:49,688 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,689 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 11:21:49,689 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:49,689 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:21:49,689 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:49,689 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:49,689 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:21:49,689 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 11:21:49,968 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:21:49,968 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:21:49,968 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:21:49,968 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:49,969 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 11:21:49,969 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 11:21:50,208 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:21:50,208 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:21:50,208 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 11:21:50,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,208 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 11:21:50,270 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:50,270 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,271 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,280 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,332 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:50,429 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:21:50,429 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,429 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 11:21:50,429 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,429 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:50,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 11:21:50,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:50,431 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:50,431 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:21:50,432 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:50,432 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,432 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,432 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 11:21:50,433 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:50,433 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 11:21:50,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 11:21:50,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,435 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 11:21:50,436 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:50,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:50,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:21:50,436 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:50,438 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,465 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:21:50,465 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 11:21:50,466 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 11:21:50,466 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 11:21:50,466 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 11:21:50,467 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#97780231] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 11:21:50,468 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 11:21:50,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 11:21:50,468 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 11:21:50,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,468 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 11:21:50,468 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Follower 11:21:50,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:21:50,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,478 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 11:21:50,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,480 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:50,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:50,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:50,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:50,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:50,482 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:50,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:21:50,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 11:21:50,484 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,485 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:21:50,485 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,485 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 11:21:50,485 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,487 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,487 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,487 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:21:50,487 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,487 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,487 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,487 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,488 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:21:50,488 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:50,534 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-716593625] 11:21:50,534 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 11:21:50,534 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306] 11:21:50,535 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:50,537 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:50,540 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 11:21:50,543 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#426365078] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,546 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 11:21:50,547 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 11:21:50,547 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,547 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@27d4a1ce to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43b565c3 11:21:50,548 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@70acd5c2 11:21:50,548 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 11:21:50,548 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,548 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,549 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,554 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 11:21:50,554 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,555 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709032910555, None)) 11:21:50,555 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032910554,0,0)) 11:21:50,555 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3fdaf4b2 11:21:50,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43b565c3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4ff83f97 11:21:50,556 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556] 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556] 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556] 11:21:50,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-2133973718] 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,557 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testLeadershipTransferOnShutdown commit rateLimit adjusted to 155.8689248333021 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-2133973718] 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-2133973718] 11:21:50,557 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,557 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-906089221] 11:21:50,557 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 11:21:50,557 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 11:21:50,557 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-906089221] 11:21:50,557 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 11:21:50,557 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 11:21:50,557 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying 2 components for commit 11:21:50,557 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard cars 11:21:50,557 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,557 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 11:21:50,557 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard people 11:21:50,557 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 11:21:50,558 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 11:21:50,558 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 11:21:50,558 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1826448997] 11:21:50,558 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1713372583 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,558 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:50,558 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:21:50,558 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:21:50,558 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 11:21:50,558 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#2005370913 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,558 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-2133973718] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,558 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:21:50,558 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-716593625] 11:21:50,558 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1713372583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 2 ] 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 2 batched modifications, ready: true 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] for shard people 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 1 ] 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 1 batched modifications, ready: true 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] 11:21:50,559 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 11:21:50,559 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:50,559 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:50,560 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:50,560 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:50,560 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,560 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,560 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 11:21:50,560 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 11:21:50,560 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 11:21:50,560 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-860451042] 11:21:50,560 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,560 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,560 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 11:21:50,560 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:21:50,560 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 11:21:50,561 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:50,561 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:50,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 11:21:50,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 11:21:50,561 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,561 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,561 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,561 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,561 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 11:21:50,561 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:21:50,561 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 11:21:50,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 11:21:50,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 2 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 2 operations, client version 13 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,562 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 1 operations, client version 13 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,564 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 11:21:50,564 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 11:21:50,564 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,564 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,564 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:21:50,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:50,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,565 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,565 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:21:50,565 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:50,565 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,565 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:21:50,565 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 11:21:50,565 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 11:21:50,565 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,565 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 11:21:50,565 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:50,566 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,566 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 11:21:50,566 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,567 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=6821247} is already disabled 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=6030369} is already disabled 11:21:50,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,569 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 11:21:50,571 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:21:50,571 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,571 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:21:50,571 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:50,571 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,571 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,571 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 11:21:50,571 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 11:21:50,572 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 11:21:50,572 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 11:21:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 11:21:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 11:21:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:21:50,573 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 11:21:50,574 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:50,574 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 11:21:50,574 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:21:50,574 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 11:21:50,574 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,574 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 11:21:50,574 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:21:50,610 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:50,610 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,610 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:50,611 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:50,611 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 11:21:50,611 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:21:50,611 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying 1 components for commit 11:21:50,611 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying transaction for shard cars 11:21:50,611 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 directCommit called 11:21:50,611 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 sending 1 batched modifications, ready: true 11:21:50,611 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 11:21:50,611 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-716593625] 11:21:50,611 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 11:21:50,612 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:50,612 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:50,615 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,615 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 of 1 operations, client version 13 11:21:50,615 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,615 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 ready 11:21:50,615 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 is scheduled for canCommit step 11:21:50,662 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:50,663 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 11:21:50,663 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-716593625] 11:21:50,663 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 11:21:50,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:21:50,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:21:50,664 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164] 11:21:50,664 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 canCommit 11:21:50,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 11:21:50,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 11:21:50,664 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164)] resolved 11:21:50,664 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306)] resolved 11:21:50,664 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: successfully resolved all cohort actors 11:21:50,664 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finishCanCommit 11:21:50,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4cf9079f 11:21:50,664 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164)] 11:21:50,664 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164)] 11:21:50,665 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:21:50,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,667 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:21:50,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,667 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:21:50,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 11:21:50,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:50,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 11:21:50,668 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,668 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,669 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:50,669 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306)] 11:21:50,669 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306)] 11:21:50,670 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,670 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,670 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 11:21:50,670 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:50,670 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 11:21:50,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:50,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit returning result: true 11:21:50,671 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 commit 11:21:50,671 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finish commit 11:21:50,671 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164)] 11:21:50,671 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-253855164)] 11:21:50,672 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306)] 11:21:50,672 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1924938306)] 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 169 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 validated 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: success 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 candidate ready 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - candidate rootPath: / 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - messages: [] 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 prepared 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 scheduled for commit step 11:21:50,673 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 11:21:50,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 11:21:50,674 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:50,674 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,674 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 11:21:50,674 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:50,674 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,674 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 11:21:50,674 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 228 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 11:21:50,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 submitted to persistence 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 11:21:50,675 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:21:50,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 11:21:50,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 11:21:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:50,677 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-people-testLeadershipTransferOnShutdown%23-1924938306$g] 11:21:50,677 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown%23-253855164$f] 11:21:50,678 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#192339235] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1609515075] with name member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 11:21:50,678 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: commit - successfulFromPrevious: [] 11:21:50,678 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$c] 11:21:50,678 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#192339235] 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4cf9079f 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 11:21:50,678 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 11:21:50,678 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:50,678 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 122.5 μs 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 11:21:50,678 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#192339235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,678 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1609515075] 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:50,679 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 11:21:50,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 11:21:50,679 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 57.65 μs 11:21:50,679 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,679 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1609515075] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,679 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 48.21 μs 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 11:21:50,679 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1551611975] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 11:21:50,680 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 11:21:50,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 11:21:50,681 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#436801672] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 11:21:50,681 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#436801672] 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 11:21:50,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 11:21:50,681 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,682 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 70.57 μs 11:21:50,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1551611975] 11:21:50,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 46.19 μs 11:21:50,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 36.50 μs 11:21:50,682 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 11:21:50,682 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 11:21:50,682 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,682 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,683 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,683 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:21:50,683 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:21:50,683 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 11:21:50,683 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 11:21:50,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-475: Closing 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,686 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:21:50,686 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 11:21:50,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit succeeded 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 11:21:50,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 11:21:50,689 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 canCommit 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 successfully completed direct commit 11:21:50,689 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:21:50,690 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cannot append entries because sender's term 4 is less than 5 11:21:50,690 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:21:50,690 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:21:50,690 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,690 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:50,690 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:50,691 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:21:50,691 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:21:50,691 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 11:21:50,691 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 11:21:50,691 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,691 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:21:50,692 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,692 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,692 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,692 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 11:21:50,692 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,693 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,693 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#916935814] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 11:21:50,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 11:21:50,695 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#916935814] 11:21:50,695 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,695 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#916935814] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,695 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 86.14 μs 11:21:50,695 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,695 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,695 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,695 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,696 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-3-shard-cars-testLeadershipTransferOnShutdown in 31.43 ms 11:21:50,696 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,696 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 11:21:50,696 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,696 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,696 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,696 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:21:50,696 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:21:50,696 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 11:21:50,696 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-906089221] 11:21:50,696 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:50,697 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Queuing TransactionOperation 11:21:50,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:21:50,697 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:21:50,697 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 11:21:50,697 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] found - trying create transaction 11:21:50,697 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] 11:21:50,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 11:21:50,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:50,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 11:21:50,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 11:21:50,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Creating transaction : member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 11:21:50,705 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#-504105656] 11:21:50,705 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#-504105656, transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0, version=13] 11:21:50,705 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:50,706 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#-504105656)] 11:21:50,707 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 ReadData operation succeeded 11:21:50,707 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,707 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#426365078] 11:21:50,708 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 11:21:50,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:50,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@67c4ecc1] 11:21:50,708 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@67c4ecc1] 11:21:50,708 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556)] for shard cars 11:21:50,708 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:21:50,708 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5a73ba33 11:21:50,708 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,708 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 11:21:50,708 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 11:21:50,708 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 11:21:50,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b309291 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b309291 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,709 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,710 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1544442556] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 11:21:50,715 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,715 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:21:50,715 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1696512286] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 11:21:50,715 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1696512286] 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 11:21:50,715 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,716 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 151.8 μs 11:21:50,716 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1696512286] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:50,715 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:21:50,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 11:21:50,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 11:21:50,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 11:21:50,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 11:21:50,723 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 11:21:50,723 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,723 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 11:21:50,723 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:21:50,723 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 65.77 μs 11:21:50,723 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:21:50,724 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,724 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,724 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 11:21:50,724 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,724 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,724 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 6 11:21:50,724 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,725 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 6 11:21:50,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,726 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:21:50,726 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:21:50,726 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 5 - switching to Follower 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=null 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 2, lastTerm: 4 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 6 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-479: Closing 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,727 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:21:50,727 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 11:21:50,728 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 11:21:50,728 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,728 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 11:21:50,728 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,729 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 20.02 ms 11:21:50,729 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 11:21:50,729 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:21:50,729 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 11:21:50,729 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,729 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:21:50,729 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 11:21:50,729 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 11:21:50,730 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 11:21:50,730 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:50,731 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:21:50,731 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 11:21:50,731 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 11:21:50,731 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e2f12f0 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e2f12f0 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 11:21:50,731 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:21:50,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,732 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:50,732 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:50,732 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 11:21:50,734 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:21:50,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:50,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:50,735 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 11:21:50,735 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 11:21:50,735 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,735 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,735 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-3-shard-people-testLeadershipTransferOnShutdown 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 11:21:50,736 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:21:50,736 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-474: Closing 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 11:21:50,737 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:21:50,737 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 11:21:50,738 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 11:21:50,738 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:21:50,739 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 11:21:50,739 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,739 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 7.688 ms 11:21:50,739 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 11:21:50,739 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 11:21:50,739 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 11:21:50,739 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:21:50,739 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:21:50,740 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 11:21:50,740 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 11:21:50,740 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:21:50,740 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:21:50,740 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 11:21:50,740 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:21:50,740 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:50,740 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:21:50,741 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:50,741 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:50,751 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:50,752 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:50,752 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:50,752 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:50,762 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:21:50,763 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 11:21:50,763 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 11:21:50,763 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-480: Closing 11:21:50,763 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-481: Closing 11:21:50,763 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:21:50,763 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:21:50,763 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 11:21:50,764 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:50,764 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:51,760 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1680027953749489702] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:51,778 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:51,778 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:51,780 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7965598113198990047] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:51,784 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:51.786] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:51,800 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:51,800 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:21:51.808] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:51,829 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:51,836 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8645254980374722786] 11:21:51,836 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:51,838 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:51,838 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:51,838 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:51,838 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:51,856 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:51,856 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:51,856 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:51,861 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:21:51,868 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2213842404505358768] 11:21:51,868 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:51,870 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:51,870 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:51,872 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:51,872 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:51,904 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:51,923 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7099478203265329357] 11:21:51,923 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:51,923 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:51,923 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:51,934 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:51,935 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:51,937 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 11:21:51,942 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:51,942 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:51,944 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,944 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:21:51,944 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 11:21:51,944 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:51,944 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7eab2a3e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a905558 11:21:51,945 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6eafccd2 11:21:51,945 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709032911945, None)) 11:21:51,946 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032911944,0,0)) 11:21:51,946 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a905558 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@fb11f89 11:21:51,946 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 11:21:51,946 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:51,947 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:51,948 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:51,948 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2a9101fd 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230] 11:21:51,949 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:21:51,950 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,950 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 11:21:51,950 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:21:51,950 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:21:51,950 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,955 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-426681573 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,955 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1963332927] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,955 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-426681573] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,955 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1046906189] 11:21:51,955 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1046906189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,957 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#133729631] 11:21:51,957 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:21:51,957 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#133729631] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,957 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1963332927 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,958 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:51,958 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,963 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:21:51,965 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 11:21:51,965 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,966 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 11:21:51,966 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:51,966 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5258b57d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6e074301 11:21:51,966 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,966 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@99d0855 11:21:51,966 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709032911966, None)) 11:21:51,966 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:51,966 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,966 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,966 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032911965,0,0)) 11:21:51,966 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6b4b9d85 11:21:51,966 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6e074301 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@611469eb 11:21:51,966 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 11:21:51,966 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:51,966 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,967 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,967 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,967 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:51,967 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,967 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,967 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,967 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1998494255] 11:21:51,967 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:21:51,967 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,968 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:21:51,968 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1998494255] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,968 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:21:51,968 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#472612475 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,968 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:21:51,968 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-198432756 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,968 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1411515504] 11:21:51,968 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1969876266] 11:21:51,968 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#472612475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:51,975 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:51,975 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:51,975 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:51,980 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:51,980 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:51,980 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:21:51,980 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:51,980 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 11:21:51,980 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 11:21:51,980 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:21:51,984 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-198432756] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,984 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1411515504] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,984 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1969876266] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:51,984 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 11:21:51,984 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 11:21:51,984 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:21:51,984 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:21:51,984 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:21:51,984 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:51,984 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:51,986 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:21:51,986 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:21:51,986 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:21:51,986 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:21:51,986 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:21:51,988 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:51,991 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:52,17 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:52,17 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:52,17 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:52,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:52,225 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:21:52,226 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:52,234 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:21:52,234 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:21:52,512 AM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:52,524 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:21:52,525 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,525 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 11:21:52,542 AM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:52,544 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:21:52,544 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,544 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 11:21:52,754 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:21:52,754 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,754 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 11:21:52,794 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:21:52,794 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,794 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 11:21:52,855 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:52,855 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,856 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,863 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,944 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:52,984 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:21:52,984 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,984 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 11:21:52,984 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,986 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:52,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:52,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:52,988 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:21:52,988 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:52,988 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:52,988 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:21:52,988 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 11:21:52,989 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:21:52,989 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:52,989 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:52,989 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:52,989 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:21:52,989 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:52,989 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:52,989 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 11:21:52,989 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:52,989 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:52,989 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:52,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 11:21:52,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:53,104 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:21:53,104 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,104 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 11:21:53,104 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:53,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:21:53,106 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 11:21:53,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:53,106 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:53,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 11:21:53,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:53,107 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,107 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:53,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:53,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:53,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:21:53,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:53,111 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:53,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:21:53,111 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 11:21:53,113 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,114 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,114 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 11:21:53,115 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 11:21:53,115 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:53,116 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:21:53,116 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:53,116 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:53,116 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:53,116 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:53,116 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:53,116 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:53,116 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:21:53,116 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:21:53,116 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:21:53,172 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 11:21:53,173 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230] 11:21:53,174 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,174 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,174 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,174 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,174 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,175 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,175 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,175 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionForwardedToLeaderAfterRetry commit rateLimit adjusted to 155.8689248333021 11:21:53,175 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,176 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,176 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 11:21:53,176 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:53,176 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,176 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,176 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:21:53,176 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 11:21:53,176 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying 2 components for commit 11:21:53,176 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,176 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard cars 11:21:53,176 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 11:21:53,176 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard people 11:21:53,176 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,176 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 canCommit 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 11:21:53,176 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,179 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 11:21:53,179 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 11:21:53,179 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:53,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:53,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,180 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 11:21:53,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,181 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 11:21:53,181 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,181 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 11:21:53,181 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,182 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,182 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,182 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,182 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,182 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:53,183 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,183 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,183 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:53,183 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,183 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,183 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,183 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 11:21:53,183 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,183 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 11:21:53,183 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,183 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 11:21:53,183 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:53,183 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:53,183 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] resolved 11:21:53,183 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 11:21:53,183 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,183 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] resolved 11:21:53,184 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: successfully resolved all cohort actors 11:21:53,184 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finishCanCommit 11:21:53,184 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,184 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,184 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 11:21:53,184 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=5694221} is already disabled 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=5764043} is already disabled 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:53,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,185 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,186 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:53,186 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:21:53,186 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,186 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:21:53,188 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,188 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,188 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 11:21:53,188 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,188 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 11:21:53,188 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:53,188 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit returning result: true 11:21:53,188 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 commit 11:21:53,188 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finish commit 11:21:53,188 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,188 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,188 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,188 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,189 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,189 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 11:21:53,190 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 11:21:53,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 11:21:53,191 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:53,191 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:53,191 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 11:21:53,191 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 11:21:53,191 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,191 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,191 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:53,191 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:53,191 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=183} 11:21:53,191 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=179} 11:21:53,191 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:53,191 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:53,191 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:21:53,191 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:21:53,192 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,192 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:53,192 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-1535805944$e] 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-1395818230$f] 11:21:53,193 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2044198436] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:21:53,193 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1802403761] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:21:53,193 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 11:21:53,193 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 11:21:53,193 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:53,193 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:53,193 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2044198436] 11:21:53,193 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1802403761] 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 70.55 μs 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 109.1 μs 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2044198436] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:53,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1802403761] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:53,194 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit succeeded 11:21:53,196 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,196 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,196 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,197 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-502506719] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 11:21:53,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-502506719] 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-502506719] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 59.55 μs 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#96957190] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 11:21:53,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#96957190] 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 33.65 μs 11:21:53,198 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#96957190] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:53,199 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,199 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 11:21:53,206 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1998494255] 11:21:53,206 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1998494255] 11:21:53,206 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,206 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:53,206 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:53,206 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 11:21:53,206 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:21:53,206 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying 2 components for commit 11:21:53,206 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard cars 11:21:53,206 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 11:21:53,206 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,206 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard people 11:21:53,206 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 11:21:53,206 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 11:21:53,206 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,206 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 canCommit 11:21:53,208 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 11:21:53,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 11:21:53,208 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 11:21:53,208 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,208 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 11:21:53,209 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 11:21:53,209 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] resolved 11:21:53,209 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 11:21:53,210 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 11:21:53,210 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] resolved 11:21:53,210 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: successfully resolved all cohort actors 11:21:53,210 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finishCanCommit 11:21:53,210 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,210 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,210 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 11:21:53,211 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,211 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,211 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 11:21:53,211 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,211 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 11:21:53,212 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:53,212 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,212 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,213 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,213 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,213 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 11:21:53,213 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,213 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 11:21:53,214 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:53,214 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit returning result: true 11:21:53,214 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,215 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:53,215 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:53,215 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,215 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 11:21:53,215 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 11:21:53,215 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:53,215 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,215 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 11:21:53,215 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying 2 components for commit 11:21:53,215 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard cars 11:21:53,215 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 11:21:53,215 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 11:21:53,216 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 11:21:53,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,216 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard people 11:21:53,216 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 11:21:53,216 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 11:21:53,216 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 11:21:53,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,217 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,217 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,217 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,217 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,217 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,217 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,217 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,218 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,218 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,218 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 11:21:53,218 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,218 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,218 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 11:21:53,218 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 2 batched modifications, ready: false 11:21:53,218 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 11:21:53,219 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 11:21:53,219 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car5}] 11:21:53,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,221 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 11:21:53,221 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 11:21:53,221 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 11:21:53,221 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 11:21:53,230 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 11:21:53,230 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 2 batched modifications, ready: false 11:21:53,230 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,230 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car6}] 11:21:53,230 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,231 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:53,231 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:53,231 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 11:21:53,231 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car7}] 11:21:53,231 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,231 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:53,231 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:53,231 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 11:21:53,231 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,231 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Queuing TransactionOperation 11:21:53,231 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,231 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,232 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,232 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,233 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,233 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,233 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#1265637480] 11:21:53,234 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#1265637480] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:53,234 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 11:21:53,234 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#1265637480, transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, version=13] 11:21:53,234 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}] 11:21:53,282 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,282 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,283 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,283 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,283 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,283 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:53,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-489: Closing 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-488: Closing 11:21:53,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:53,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:53,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:21:53,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,284 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 11:21:53,284 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:21:53,284 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 11:21:53,285 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:53,285 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:53,285 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 11:21:53,285 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:21:53,285 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:53,285 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:53,285 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 11:21:53,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,334 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,334 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,334 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 canCommit 11:21:53,334 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Readying 1 components for commit 11:21:53,334 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Readying transaction for shard cars 11:21:53,334 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 directCommit called 11:21:53,334 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 1 batched modifications, ready: true 11:21:53,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,334 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Readying 1 components for commit 11:21:53,334 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Readying transaction for shard cars 11:21:53,334 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 directCommit called 11:21:53,334 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 sending 1 batched modifications, ready: true 11:21:53,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,334 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Readying 1 components for commit 11:21:53,334 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Readying transaction for shard cars 11:21:53,334 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 directCommit called 11:21:53,334 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 sending 1 batched modifications, ready: true 11:21:53,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#1265637480)] 11:21:53,335 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 11:21:53,335 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] resolved 11:21:53,335 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] resolved 11:21:53,335 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: successfully resolved all cohort actors 11:21:53,335 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 finishCanCommit 11:21:53,335 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,335 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,336 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,336 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:21:53,336 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,336 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:21:53,336 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,336 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:53,336 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 11:21:53,336 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:53,336 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:53,336 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 11:21:53,336 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:53,340 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, ready=true, participatingShardNames=null, totalMessagesSent=3, modifications size=1] to retry 11:21:53,341 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, ready=true, participatingShardNames=null, totalMessagesSent=1, modifications size=1] to retry 11:21:53,341 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,341 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,341 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to retry 11:21:53,341 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: handleForwardedReadyTransaction for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,341 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message ForwardedReadyTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, transaction=ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, closed=false, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}], operation=WRITE}}}}}}}]}, doImmediateCommit=true, participatingShardNames=null, txnClientVersion=13] to retry 11:21:53,385 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,385 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,385 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,385 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 11:21:53,385 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:21:53,385 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,386 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:53,386 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:53,387 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,388 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:21:53,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:21:53,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:21:53,390 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:21:53,390 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:53,390 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:21:53,390 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:21:53,390 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:21:53,390 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:53,390 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:21:53,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:53,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:53,390 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:21:53,390 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:21:53,391 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,391 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 pending transaction messages to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, ready=true, participatingShardNames=null, totalMessagesSent=3, modifications size=1] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, ready=true, participatingShardNames=null, totalMessagesSent=1, modifications size=1] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message ForwardedReadyTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, transaction=ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, closed=false, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}], operation=WRITE}}}}}}}]}, doImmediateCommit=true, participatingShardNames=null, txnClientVersion=13] 11:21:53,393 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,393 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 BatchedModifications to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 1 BatchedModifications to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CanCommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: handleForwardedReadyTransaction for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,394 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,395 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:53,395 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:53,395 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:53,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:21:53,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:53,396 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 11:21:53,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 of 5 operations, client version 13 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 ready 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 is scheduled for canCommit step 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 of 1 operations, client version 13 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 ready 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 is scheduled for canCommit step 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 is scheduled for canCommit step 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 of 1 operations, client version 13 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 ready 11:21:53,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 is scheduled for canCommit step 11:21:53,403 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 11:21:53,403 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.datastore.messages.BatchedModificationsReply] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] to Actor[akka://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:53,435 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 11:21:53,436 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1998494255] 11:21:53,436 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 commit 11:21:53,436 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finish commit 11:21:53,436 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,436 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,436 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,436 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:21:53,436 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:53,437 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:21:53,437 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:21:53,437 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,437 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:21:53,437 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:21:53,437 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:53,437 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:53,437 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:21:53,438 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,439 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Adding message CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to retry 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,439 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:21:53,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:21:53,440 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:21:53,440 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:21:53,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:53,440 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:21:53,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:21:53,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:21:53,440 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:53,441 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:53,441 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:53,441 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:21:53,441 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:21:53,441 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 candidate ready 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - candidate rootPath: / 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - messages: [] 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 prepared 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 77 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 validated 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: success 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,441 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}]]] 11:21:53,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 submitted to persistence 11:21:53,442 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:53,442 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,442 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}]]] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=77} 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 pending transaction messages to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Retrying message CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CanCommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,444 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 11:21:53,444 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 11:21:53,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:21:53,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:53,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 11:21:53,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Evaluating tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 for canCommit - preceding participating shard names [cars] 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Skipping pending transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 in state CAN_COMMIT_COMPLETE 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Not moving tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 - cannot proceed with canCommit 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,448 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 candidate ready 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - candidate rootPath: / 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - messages: [] 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 prepared 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 validated 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: success 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}]]] 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 submitted to persistence 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=77} 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit - successfulFromPrevious: [] 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-1535805944$t] 11:21:53,450 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 11:21:53,450 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:53,450 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:21:53,450 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:53,450 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit returning result: true 11:21:53,450 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,450 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 26.91 μs 11:21:53,450 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 11:21:53,451 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}]]] 11:21:53,451 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,451 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:53,451 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 11:21:53,451 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=77} 11:21:53,451 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:21:53,451 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 11:21:53,451 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,451 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:21:53,451 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 22.02 μs 11:21:53,452 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=184} 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit - successfulFromPrevious: [] 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-1395818230$u] 11:21:53,452 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit succeeded 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:21:53,452 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,452 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:21:53,452 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 commit 11:21:53,452 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 finish commit 11:21:53,452 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,452 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944)] 11:21:53,452 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 99.42 μs 11:21:53,452 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,452 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-1395818230)] 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:21:53,458 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=184} 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:21:53,458 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 77.33 μs 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 candidate ready 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - candidate rootPath: / 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 candidate ready 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - candidate rootPath: / 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - messages: [] 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 prepared 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - messages: [] 11:21:53,459 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 prepared 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 validated 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: success 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 candidate ready 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: canCommit - candidate rootPath: / 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: canCommit - messages: [] 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: preCommit - successfulFromPrevious: [] 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 prepared 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 scheduled for commit step 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}]]] 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 validated 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: success 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 submitted to persistence 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 candidate ready 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: canCommit - candidate rootPath: / 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: canCommit - messages: [] 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 prepared 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 scheduled for commit step 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 validated 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: success 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 candidate ready 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: canCommit - candidate rootPath: / 11:21:53,461 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}]]] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: canCommit - messages: [] 11:21:53,461 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: preCommit - successfulFromPrevious: [] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 prepared 11:21:53,461 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,461 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=222} 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 scheduled for commit step 11:21:53,461 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 11:21:53,461 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 submitted to persistence 11:21:53,461 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 324 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 submitted to persistence 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 submitted to persistence 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 1 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}], SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}]]] 11:21:53,461 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 submitted to persistence 11:21:53,462 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 11:21:53,463 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 11:21:53,463 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 11:21:53,463 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 11:21:53,463 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}], SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}]]] 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 4 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=324} 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 5 11:21:53,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 3, nextIndex: 4 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=222} 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit - successfulFromPrevious: [] 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-1395818230$x] 11:21:53,467 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,468 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 79.52 μs 11:21:53,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 11:21:53,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 2 11:21:53,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 11:21:53,469 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:21:53,469 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=236} 11:21:53,469 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,469 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit - successfulFromPrevious: [] 11:21:53,469 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,469 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-1535805944$w] 11:21:53,470 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,470 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit succeeded 11:21:53,470 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 11:21:53,470 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 85.59 μs 11:21:53,470 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 11:21:53,470 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 11:21:53,470 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 11:21:53,479 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 canCommit 11:21:53,479 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 11:21:53,479 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 successfully completed direct commit 11:21:53,479 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:21:53,479 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 11:21:53,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 11:21:53,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:21:53,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=324} 11:21:53,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,479 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 canCommit 11:21:53,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: commit - successfulFromPrevious: [] 11:21:53,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$p] 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data Simple{size=236} 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: commit - successfulFromPrevious: [] 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$q] 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=236} 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: commit - successfulFromPrevious: [] 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry_shard-cars-member-2%3Adatastore-testTransactionForwardedToLeaderAfterRetry%400%3A5_15%231265637480$r] 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:21:53,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 11:21:53,482 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 3 11:21:53,482 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 11:21:53,482 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 11:21:53,482 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:21:53,482 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=222} 11:21:53,482 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,482 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 11:21:53,482 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 2, lastIndex(): 6 11:21:53,482 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 11:21:53,482 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:21:53,482 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=236} 11:21:53,483 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 11:21:53,483 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 11:21:53,483 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:21:53,483 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,483 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 110.0 μs 11:21:53,483 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,483 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 11:21:53,483 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 54.99 μs 11:21:53,483 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:21:53,483 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,483 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=324} 11:21:53,483 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 79.99 μs 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 successfully completed direct commit 11:21:53,484 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 modifications completed with CommitTransactionReply [version=13] 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 modifications completed with CommitTransactionReply [version=13] 11:21:53,484 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 canCommit 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 successfully completed direct commit 11:21:53,484 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 61.18 μs 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,484 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 51.36 μs 11:21:53,484 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1535805944] 11:21:53,484 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 70.64 μs 11:21:53,484 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 11:21:53,484 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 11:21:53,484 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data Simple{size=236} 11:21:53,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:21:53,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 11:21:53,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 11:21:53,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=236} 11:21:53,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:21:53,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 11:21:53,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 11:21:53,485 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,486 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 46.70 μs 11:21:53,486 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:21:53,486 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 51.71 μs 11:21:53,486 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1912632273] 11:21:53,486 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:21:53,487 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,489 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-170972188] 11:21:53,489 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,489 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,489 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 11:21:53,489 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:53,490 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,490 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,490 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 11:21:53,490 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 11:21:53,490 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 11:21:53,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:53,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 11:21:53,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,495 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#515654896] 11:21:53,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 11:21:53,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:21:53,496 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=7, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,496 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#515654896, transactionId=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] 11:21:53,496 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:21:53,496 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#515654896)] 11:21:53,497 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,497 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,497 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 7, nextIndex: 8 11:21:53,497 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 11:21:53,497 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 11:21:53,497 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 11:21:53,497 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,498 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,498 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,498 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=3255095} is already disabled 11:21:53,498 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 6 for term 5 11:21:53,498 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:21:53,498 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=7 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 7 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 7 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 6 for term 5 11:21:53,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=7 11:21:53,500 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ReadData operation succeeded 11:21:53,500 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#235741347] 11:21:53,500 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 11:21:53,501 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:21:53,501 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,501 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:21:53,502 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 11:21:53,502 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 11:21:53,502 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 4 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:21:53,504 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-1239056002] 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:21:53,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:53,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-1239056002, transactionId=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] 11:21:53,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 11:21:53,504 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-1239056002)] 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=2494617} is already disabled 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:21:53,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 11:21:53,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 11:21:53,508 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ReadData operation succeeded 11:21:53,508 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,508 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,508 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,509 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 11:21:53,509 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 11:21:53,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 11:21:53,509 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,509 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,509 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:53,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:21:53,509 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:21:53,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:21:53,509 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 11:21:53,509 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:21:53,509 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,509 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:21:53,510 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 11:21:53,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,512 AM [cluster-test-akka.actor.default-dispatcher-26] [ERROR] OneForOneStrategy - Cannot invoke "org.opendaylight.controller.cluster.access.client.AbstractClientActorBehavior.persistenceId()" because "this.currentBehavior" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.access.client.AbstractClientActorBehavior.persistenceId()" because "this.currentBehavior" is null at org.opendaylight.controller.cluster.access.client.AbstractClientActor.persistenceId(AbstractClientActor.java:31) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:21:53,515 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:53,515 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:53,521 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:53,521 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:53,521 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:53,521 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:53,523 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:53,546 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:21:53,547 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:21:53,547 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-490: Closing 11:21:53,548 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:21:53,548 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:21:53,548 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-491: Closing 11:21:53,548 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:21:53,548 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:21:53,548 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:53,548 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:54,534 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8645254980374722786] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:54,564 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2213842404505358768] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:54,564 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:54,564 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:21:54.569] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:54,577 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:54,577 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:54,579 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:54.580] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:54,603 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:54,610 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2751694345857695674] 11:21:54,610 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:54,617 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:54,617 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:54,617 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:54,617 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:54,621 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:54,621 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:54,621 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:54,624 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:54,636 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7596581435662200865] 11:21:54,636 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:54,640 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:54,640 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:54,645 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:54,645 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:54,655 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:54,661 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [9137837844702606213] 11:21:54,661 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:54,667 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:54,667 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:54,667 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:54,667 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:54,672 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:54,673 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:54,674 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,684 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,689 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-48392666] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,704 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,704 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,704 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 11:21:54,704 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:54,704 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@a5243aa to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fe8f0d9 11:21:54,704 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@84aaa10 11:21:54,704 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709032914704, None)) 11:21:54,704 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:54,704 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,705 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032914703,0,0)) 11:21:54,705 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3c191338 11:21:54,705 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fe8f0d9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4ee679d4 11:21:54,705 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 11:21:54,705 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:54,708 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:54,710 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:54,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,720 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2109363660] 11:21:54,720 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 11:21:54,720 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 11:21:54,721 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#1173685863] 11:21:54,721 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2109363660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,721 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-405613874 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,721 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#1173685863] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,723 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-240391114] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,723 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:54,723 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:54,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:21:54,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:21:54,723 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 11:21:54,724 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 11:21:54,724 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 11:21:54,724 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,724 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 11:21:54,725 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-405613874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,732 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,732 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,732 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 11:21:54,732 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:54,732 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@14920325 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2eb08106 11:21:54,732 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2a8f1008 11:21:54,733 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709032914732, None)) 11:21:54,733 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032914731,0,0)) 11:21:54,733 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4c3f751b 11:21:54,733 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2eb08106 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5e28862f 11:21:54,733 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 11:21:54,733 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:54,733 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,740 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 11:21:54,740 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2111623153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,740 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1995514526 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,740 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-2116153290] 11:21:54,741 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1995514526] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,741 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 11:21:54,741 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-2116153290] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:54,741 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:54,741 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:54,741 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:21:54,741 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:21:54,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 11:21:54,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 11:21:54,742 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 11:21:54,742 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,742 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 11:21:54,753 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:54,755 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:54,778 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:54,780 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:54,781 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:54,781 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,781 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,781 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 closeTransaction called 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-1239056002)] 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 closeTransaction called 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#377058369)] 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 closeTransaction called 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#-504105656)] 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 closeTransaction called 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#515654896)] 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#1622313767)] 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 closeTransaction called 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#1056756884)] 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 closeTransaction called 11:21:54,782 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1663025263)] 11:21:54,783 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-48392666] 11:21:54,784 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 11:21:54,784 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2109363660] 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:21:54,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:21:54,959 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 11:21:54,960 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 11:21:55,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 11:21:55,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,279 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 11:21:55,352 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:55,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 11:21:55,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,559 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 11:21:55,630 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:55,630 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,631 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,672 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:55,672 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,673 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:55,673 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,673 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,681 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:55,878 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 11:21:55,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,879 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 11:21:55,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,881 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2109363660] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:55,881 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:55,881 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 11:21:55,881 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:55,881 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:55,881 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 11:21:55,882 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:55,882 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,882 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:55,883 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:55,883 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 11:21:55,884 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 11:21:55,884 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:21:55,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:55,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,886 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 11:21:55,886 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2109363660] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:55,886 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:21:55,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:55,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:55,886 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:21:55,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 11:21:55,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:55,892 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:55,894 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:55,895 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,900 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-964955809] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:55,901 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,902 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 11:21:55,902 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:55,903 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7e323592 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@8a1ddb8 11:21:55,903 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2db2c171 11:21:55,903 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709032915903, None)) 11:21:55,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:55,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:55,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,904 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:55,904 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,904 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032915902,0,0)) 11:21:55,904 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@8a1ddb8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7ff6b86f 11:21:55,904 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 11:21:55,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6b65f4b4 11:21:55,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:55,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,905 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 11:21:55,905 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-240391114] 11:21:55,905 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Queuing TransactionOperation 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1488595582] 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,905 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1488595582] 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1488595582] 11:21:55,905 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 11:21:55,905 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 11:21:55,906 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 11:21:55,906 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 11:21:55,906 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars 11:21:55,906 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 11:21:55,906 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 11:21:55,907 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1488595582] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:55,907 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-406007793 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,907 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1926649881] 11:21:55,907 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-406007793] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:55,907 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1926649881] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:55,908 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:55,908 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:55,908 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 11:21:55,908 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 11:21:55,908 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:21:55,908 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:21:55,908 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 11:21:55,908 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,908 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,908 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,909 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, disabling transaction tracking 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 11:21:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 11:21:55,917 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-956736640] 11:21:55,917 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 11:21:55,917 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 11:21:55,917 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-956736640] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:55,918 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:55,918 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,919 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-956736640, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0, version=13] 11:21:55,919 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:55,919 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-956736640)] 11:21:55,919 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 11:21:55,919 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 11:21:55,919 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:55,919 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 11:21:55,919 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 11:21:55,919 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 11:21:55,920 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:55,920 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:21:55,920 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:55,920 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:55,920 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:21:55,920 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 11:21:55,920 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 11:21:55,920 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 11:21:55,922 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 ReadData operation succeeded 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:55,922 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 0 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 11:21:55,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=6617053} is already disabled 11:21:55,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 11:21:55,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 11:21:55,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:55,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 11:21:55,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 11:21:55,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 11:21:55,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:55,924 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 11:21:55,924 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 11:21:55,924 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,924 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-496: Closing 11:21:55,925 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 11:21:55,925 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,925 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:55,925 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:55,930 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:55,930 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:55,930 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:55,930 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:55,931 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:21:55,941 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,941 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 11:21:55,942 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionRetryWithInitialAskTimeoutExOnCreateTx commit rateLimit adjusted to 155.8689248333021 11:21:55,942 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:55,942 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:55,942 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:55,942 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 11:21:55,942 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 11:21:55,942 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 11:21:55,942 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying 1 components for commit 11:21:55,942 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying transaction for shard cars 11:21:55,942 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 11:21:55,942 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 canCommit 11:21:55,943 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:21:55,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:55,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:21:55,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 11:21:55,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:21:56,453 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:21:56,482 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:21:56,482 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 11:21:56,482 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:21:56,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:56,554 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 11:21:56,554 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 11:21:56,554 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:56,554 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:56,554 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 0, lastTerm: 4 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:56,555 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:21:56,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:21:56,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:21:56,556 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 11:21:56,562 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:21:56,562 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:56,562 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:56,563 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 5 11:21:56,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 11:21:56,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 11:21:56,563 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to PreLeader 11:21:56,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:21:56,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:21:56,563 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:21:56,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:56,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Replicate message: identifier: null, logIndex: 1, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 11:21:56,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:56,563 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to PreLeader 11:21:56,564 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:56,564 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:56,564 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:56,564 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:56,564 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 11:21:56,564 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:21:56,564 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:56,564 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:56,564 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:21:56,564 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:21:56,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 11:21:56,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 4 does not match the current term 5 11:21:56,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:56,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:56,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:21:56,566 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 11:21:56,566 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 4 does not match the current term 5 11:21:56,566 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 2 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=1, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:56,567 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 1, nextIndex: 2 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 4 does not match the current term 5 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 0 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader state for ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} not found 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 1 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Received ApplyState [identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, replicatedLogEntry=SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]] - lastApplied: 1, lastIndex: 1 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 5 11:21:56,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader)_slicer-id-499: Closing 11:21:56,569 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 11:21:56,569 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 11:21:56,569 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from PreLeader to Leader 11:21:56,569 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from PreLeader to Leader 11:21:56,569 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 1 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 0 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 1 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:21:56,569 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 11:21:56,942 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2751694345857695674] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:56,943 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2751694345857695674] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:57,2 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:21:57,33 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:21:57,33 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-240391114] 11:21:57,34 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:57,34 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2111623153, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@33072cc8] 11:21:57,34 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2111623153, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@33072cc8] 11:21:57,34 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2111623153)] found - trying create transaction 11:21:57,34 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2111623153)] 11:21:57,34 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#2048339419] 11:21:57,35 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#2048339419] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:57,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#2048339419, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, version=13] 11:21:57,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:57,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 directCommit called 11:21:57,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 sending 1 batched modifications, ready: true 11:21:57,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#2048339419)] 11:21:57,36 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: handleForwardedReadyTransaction for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, client version 13 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 ready 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 validated 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: canCommit for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: success 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,36 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 candidate ready 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - candidate rootPath: / 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - messages: [] 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 prepared 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Finishing commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 191 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 11:21:57,37 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 11:21:57,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:57,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:21:57,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:21:57,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:57,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}]]] 11:21:57,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 submitted to persistence 11:21:57,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}]]] 11:21:57,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 11:21:57,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:21:57,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=191} 11:21:57,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 11:21:57,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 3 11:21:57,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:57,41 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 2, nextIndex: 3 11:21:57,41 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 2 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 1 to 2 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 2 data Simple{size=191} 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: commit - successfulFromPrevious: [] 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_shard-cars-member-2%3Adatastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx%400%3A1_19%232048339419$d] 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-803604080] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 11:21:57,42 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=2 11:21:57,43 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 11:21:57,43 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 successfully completed direct commit 11:21:57,43 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-803604080] 11:21:57,43 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 11:21:57,43 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:57,43 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-803604080] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:57,43 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 106.6 μs 11:21:57,44 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 11:21:57,44 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:57,44 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 11:21:57,44 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 11:21:57,44 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:57,44 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:21:57,44 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 11:21:57,44 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 11:21:57,44 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:57,44 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:57,44 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:57,45 AM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:21:57,45 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:57,45 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:21:57,45 AM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:21:57,45 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:57,46 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 11:21:57,46 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-964955809] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-964955809] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:57,46 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:57,46 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:21:57,46 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-964955809] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-964955809] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:57,46 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#1148708520] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1148708520] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:57,46 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:57,46 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-500: Closing 11:21:57,46 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 11:21:57,46 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:21:57,47 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:57,47 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:58,62 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7596581435662200865] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:21:58,64 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:58,65 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:58,67 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:58.068] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:58,74 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:58,74 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:21:58,75 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:21:58.077] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:58,100 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:58,107 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1906601213410280796] 11:21:58,107 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:21:58,109 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:21:58,109 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:21:58,110 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:58,110 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:58,118 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:21:58,118 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:21:58,120 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:21:58,121 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:21:58,126 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1945575438578980436] 11:21:58,126 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:21:58,128 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:58,128 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:21:58,133 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:58,133 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:58,146 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:21:58,154 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3160588943441849276] 11:21:58,154 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:21:58,156 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:21:58,156 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:21:58,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:21:58,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:21:58,166 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:58,167 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:58,168 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 11:21:58,176 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:21:58,176 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#41003131] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,184 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:21:58,184 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 11:21:58,184 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:58,184 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,184 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@75755f7e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@19525317 11:21:58,184 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5c50ce59 11:21:58,184 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,184 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:58,184 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,184 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,185 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709032918184, None)) 11:21:58,185 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032918183,0,0)) 11:21:58,185 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@69d09f2f 11:21:58,185 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@19525317 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@66071989 11:21:58,185 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 11:21:58,186 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:58,186 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,186 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:58,188 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:21:58,188 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,188 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,188 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872] 11:21:58,188 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,189 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 11:21:58,189 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:21:58,190 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 11:21:58,190 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 11:21:58,190 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:21:58,190 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:21:58,190 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,190 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,190 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1468086990] 11:21:58,191 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#952422552] 11:21:58,191 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-2139206061 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,191 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1635623340] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,191 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1468086990] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,191 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-2139206061] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,191 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#952422552] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,194 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:58,194 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,194 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,194 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541] 11:21:58,200 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:58,200 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:58,201 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 11:21:58,201 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 11:21:58,202 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1635623340 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,205 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:58,205 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 11:21:58,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:58,205 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:58,205 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 11:21:58,205 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 11:21:58,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:58,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 11:21:58,205 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:21:58,205 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:21:58,205 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:21:58,205 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:21:58,205 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 11:21:58,205 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 11:21:58,209 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#961997375] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,210 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:21:58,211 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 11:21:58,211 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,211 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:21:58,211 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:58,211 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@15754f51 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3cb5310c 11:21:58,211 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,211 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4fe1f8 11:21:58,211 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:21:58,211 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,211 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,211 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709032918211, None)) 11:21:58,212 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032918210,0,0)) 11:21:58,212 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@20402259 11:21:58,212 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3cb5310c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7f49a3e1 11:21:58,212 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 11:21:58,212 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:21:58,212 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,212 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,212 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,212 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#1093694947] 11:21:58,212 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,213 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,213 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,213 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#1557705820] 11:21:58,213 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 11:21:58,213 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#1093694947] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,213 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-462487485 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,213 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:21:58,213 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 11:21:58,214 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:21:58,214 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-462487485] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,214 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#1557705820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,215 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1690076542] 11:21:58,215 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1624354822] 11:21:58,215 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-943647069] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,218 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:21:58,225 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1690076542] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,226 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1624354822] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:58,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:58,227 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:58,227 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 11:21:58,227 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 11:21:58,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:58,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 11:21:58,227 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,227 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:21:58,227 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:21:58,228 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 11:21:58,228 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:21:58,229 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-943647069 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,230 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:21:58,230 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:21:58,230 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 11:21:58,230 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:58,230 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 11:21:58,230 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 11:21:58,230 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,230 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:21:58,230 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:21:58,230 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 11:21:58,262 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#41003131] 11:21:58,262 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 11:21:58,263 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872] 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:58,479 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 11:21:58,479 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:21:58,480 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:21:58,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 11:21:58,499 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:21:58,500 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 11:21:58,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:21:58,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:58,748 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:21:58,758 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:21:58,758 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:58,759 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:21:58,830 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:58,839 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:21:59,8 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:21:59,8 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,8 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:21:59,59 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:21:59,59 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,59 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:21:59,130 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:21:59,130 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,130 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,139 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,174 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:21:59,298 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:21:59,298 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,298 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:21:59,299 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:21:59,303 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:59,303 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 11:21:59,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 11:21:59,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,304 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:21:59,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:59,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:59,304 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 11:21:59,310 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:21:59,310 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:59,318 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:21:59,318 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,318 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:21:59,318 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,320 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:21:59,321 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:21:59,321 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:21:59,321 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 11:21:59,321 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:59,321 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:21:59,322 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:21:59,322 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:21:59,322 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:21:59,322 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:59,322 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:59,322 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:21:59,322 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 11:21:59,322 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 11:21:59,323 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 11:21:59,323 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,323 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 11:21:59,324 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:21:59,324 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:21:59,325 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:21:59,325 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:59,325 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:59,325 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 11:21:59,369 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#41003131] 11:21:59,369 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 11:21:59,369 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541] 11:21:59,369 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithMultipleShards commit rateLimit adjusted to 155.8689248333021 11:21:59,370 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,370 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#961997375] 11:21:59,370 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:21:59,370 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#961997375] 11:21:59,370 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:21:59,370 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:21:59,370 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 11:21:59,370 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 11:21:59,370 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:21:59,370 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 11:21:59,370 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:21:59,370 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:21:59,370 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 11:21:59,370 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:21:59,370 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:21:59,370 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:21:59,370 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:21:59,370 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars 11:21:59,370 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 11:21:59,370 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 11:21:59,370 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people 11:21:59,371 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 11:21:59,371 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,374 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,375 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:21:59,375 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:59,375 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,375 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 11:21:59,375 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,375 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1492285425] 11:21:59,375 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,375 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1492285425] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:59,375 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#2124796266] 11:21:59,375 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#2124796266] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:59,376 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1492285425, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 11:21:59,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:59,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:59,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 11:21:59,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#2124796266, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 11:21:59,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1492285425)] 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:21:59,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 11:21:59,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 11:21:59,378 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#2124796266)] 11:21:59,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 11:21:59,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:59,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:21:59,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 11:21:59,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:21:59,379 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:59,379 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,379 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 11:21:59,379 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 11:21:59,379 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=6841628} is already disabled 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:21:59,381 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=8030045} is already disabled 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:59,382 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,382 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,382 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,382 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,382 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,382 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 11:21:59,382 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872)] resolved 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541)] resolved 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 11:21:59,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 11:21:59,385 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872)] 11:21:59,385 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872)] 11:21:59,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:59,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:21:59,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:21:59,386 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 11:21:59,386 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:59,386 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:59,386 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:59,386 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:21:59,386 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:21:59,386 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:21:59,386 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 11:21:59,386 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:59,386 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:21:59,386 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:21:59,387 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,387 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,387 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 11:21:59,387 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:59,387 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 11:21:59,388 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:59,388 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541)] 11:21:59,388 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541)] 11:21:59,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 11:21:59,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:21:59,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 11:21:59,390 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:21:59,390 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 11:21:59,390 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 11:21:59,390 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 11:21:59,390 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872)] 11:21:59,390 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872)] 11:21:59,390 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541)] 11:21:59,390 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1234634541)] 11:21:59,391 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,391 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,391 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 11:21:59,391 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:59,391 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:59,392 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:59,392 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,392 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 11:21:59,392 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 11:21:59,392 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:59,392 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:21:59,392 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:59,392 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:59,392 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=183} 11:21:59,392 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:21:59,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 11:21:59,393 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:21:59,393 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:59,393 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 11:21:59,393 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:59,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:59,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-people-testReadWriteTransactionWithMultipleShards%231234634541$h] 11:21:59,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 11:21:59,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:21:59,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:21:59,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 11:21:59,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:59,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:21:59,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1778597075] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:21:59,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:21:59,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:21:59,394 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1778597075] 11:21:59,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:59,394 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:21:59,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:59,394 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 151.5 μs 11:21:59,395 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1778597075] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:59,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:21:59,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:21:59,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:59,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:21:59,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:21:59,395 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-cars-testReadWriteTransactionWithMultipleShards%23118931872$g] 11:21:59,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,396 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1379791580] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:21:59,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#295838219] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:21:59,396 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#295838219] 11:21:59,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:21:59,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:59,396 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:21:59,397 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 95.16 μs 11:21:59,397 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#295838219] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:59,397 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1379791580] 11:21:59,397 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:21:59,397 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 70.01 μs 11:21:59,397 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 11:21:59,397 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:21:59,397 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 11:21:59,397 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:59,397 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 11:21:59,397 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 11:21:59,397 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 11:21:59,397 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:21:59,398 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:59,398 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:59,399 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:21:59,399 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#299889331] 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#2076341404] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:21:59,399 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:21:59,399 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#299889331, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 11:21:59,400 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:21:59,400 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#299889331)] 11:21:59,400 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#2076341404] 11:21:59,400 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:21:59,400 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 70.80 μs 11:21:59,400 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#2076341404] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:21:59,402 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 11:21:59,402 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 11:21:59,402 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:21:59,402 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 11:21:59,402 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 11:21:59,402 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 11:21:59,403 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:21:59,403 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-2023210023] 11:21:59,404 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-2023210023, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 11:21:59,404 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:21:59,404 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-2023210023)] 11:21:59,406 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 11:21:59,406 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 11:21:59,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 11:21:59,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 11:21:59,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a787e7d 11:21:59,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a787e7d 11:21:59,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@21c8c4bf 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@21c8c4bf 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:21:59,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:59,406 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:59,407 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:59,407 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:59,407 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:59,407 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:59,407 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:59,406 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:59,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 11:21:59,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 11:21:59,408 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:59,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:21:59,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:21:59,408 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:21:59,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:21:59,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:21:59,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:21:59,409 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:21:59,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:21:59,409 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:21:59,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:59,409 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:21:59,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:21:59,409 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:21:59,819 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:21:59,819 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#118931872]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:21:59,819 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,819 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-508: Closing 11:21:59,819 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:21:59,820 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,820 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-507: Closing 11:21:59,820 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:21:59,820 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:21:59,820 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 11:21:59,821 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:21:59,821 AM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 11:21:59,821 AM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 11:21:59,821 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing [WARN] [02/27/2024 11:21:59.827] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-779)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:21:59.828] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-779)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:21:59.828] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-779)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:21:59.828] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-779)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:21:59,841 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 11:21:59,841 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:21:59,841 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:21:59,841 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:21:59,841 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:21:59,841 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:21:59,842 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:21:59,842 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:21:59,842 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:21:59,842 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:00,843 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1906601213410280796] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:00,862 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1945575438578980436] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:00,870 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:00,870 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:00,876 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:00.878] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:00,890 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:00,890 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:00,892 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:00.893] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:00,912 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:00,920 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8264718696054553270] 11:22:00,920 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:00,921 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:00,921 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:00,921 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:00,921 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:00,934 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:00,934 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:00,934 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:00,938 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:00,944 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8139570120025772192] 11:22:00,944 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:00,946 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:00,946 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:00,952 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:00,952 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:00,959 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:00,964 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4776395484260731276] 11:22:00,964 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:00,971 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:00,971 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:00,981 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:00,981 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:00,984 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:00,984 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:00,988 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:00,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:00,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:00,997 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:22:00,997 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:22:01,1 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:01.001] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:01,16 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:01,16 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:01,18 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:01.018] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:01,61 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:22:01,68 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1275311730176089410] 11:22:01,68 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:01,72 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:01,72 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:01,72 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:01,72 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:01,86 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:01,86 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:01,86 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:01,90 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:01,94 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [9216377097809742921] 11:22:01,94 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:01,97 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:01,97 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:01,105 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:01,105 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:01,121 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:01,130 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3766784798083955202] 11:22:01,130 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:01,136 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:01,136 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:01,141 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:01,141 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:01,143 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:01,144 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:01,146 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 11:22:01,150 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:22:01,151 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-925146075] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,159 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:01,165 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:01,172 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:01,176 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 11:22:01,176 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 11:22:01,176 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:22:01,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,176 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@489cb002 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@49a5ee29 11:22:01,176 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5b5a65e2 11:22:01,177 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:01,177 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,177 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709032921176, None)) 11:22:01,177 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032921175,0,0)) 11:22:01,177 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@49a5ee29 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3b49a0c5 11:22:01,177 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 11:22:01,177 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:01,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7449735e 11:22:01,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:01,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,178 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,178 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,178 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:01,178 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-899981295] 11:22:01,178 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 11:22:01,178 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-899981295] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,178 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1611377681 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,178 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1611377681] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,178 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#412715462] 11:22:01,179 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#412715462] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,179 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 11:22:01,179 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:01,179 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:01,179 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:01,179 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 11:22:01,179 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 11:22:01,180 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 11:22:01,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 11:22:01,180 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 11:22:01,180 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 11:22:01,180 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 11:22:01,182 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 11:22:01,185 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:22:01,187 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-833764672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,190 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 11:22:01,190 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:22:01,190 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@64a21860 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79f41aac 11:22:01,190 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7f0f67d 11:22:01,190 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709032921190, None)) 11:22:01,190 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 11:22:01,190 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032921189,0,0)) 11:22:01,190 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,190 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79f41aac to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1cfcee 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:01,191 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@250b00b3 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#33127859] 11:22:01,192 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 11:22:01,192 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#33127859] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,192 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#435993676] 11:22:01,192 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#812129310 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,192 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 11:22:01,192 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#812129310] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,192 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#435993676] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:01,193 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:01,193 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:01,193 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 11:22:01,193 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 11:22:01,193 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 11:22:01,193 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 11:22:01,193 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,193 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 11:22:01,193 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 11:22:01,193 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 11:22:01,241 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-925146075] 11:22:01,241 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 11:22:01,241 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-899981295] 11:22:01,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:01,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 11:22:01,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 11:22:01,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,418 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 11:22:01,419 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:01,419 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 11:22:01,419 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 11:22:01,419 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 11:22:01,419 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 11:22:01,419 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 11:22:01,648 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 11:22:01,648 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,649 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 11:22:01,728 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:01,968 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 11:22:01,968 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:01,969 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 11:22:02,89 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:02,89 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:02,89 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:02,89 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,89 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,89 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:02,89 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,90 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,90 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,95 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:02,95 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,95 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,95 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:02,95 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,95 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,161 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:02,198 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 11:22:02,198 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,198 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 11:22:02,198 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,203 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:02,203 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 11:22:02,203 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:02,203 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:02,204 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 11:22:02,204 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:02,204 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,204 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:02,205 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:02,205 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:02,205 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 11:22:02,205 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:02,205 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:02,205 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 11:22:02,206 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 11:22:02,206 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 11:22:02,206 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 11:22:02,206 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:02,206 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:02,206 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 11:22:02,206 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 11:22:02,207 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 11:22:02,207 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,207 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 11:22:02,207 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:02,207 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:02,207 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 11:22:02,207 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 11:22:02,208 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 11:22:02,208 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:02,259 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleTransactionsWritesInQuickSuccession commit rateLimit adjusted to 155.8689248333021 11:22:02,259 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,260 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-833764672] 11:22:02,260 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:22:02,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:02,260 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:22:02,260 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying 1 components for commit 11:22:02,260 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 11:22:02,260 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:22:02,260 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 11:22:02,260 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 canCommit 11:22:02,260 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 11:22:02,260 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars 11:22:02,260 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 11:22:02,261 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:02,261 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:22:02,261 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 directCommit called 11:22:02,261 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 11:22:02,261 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, disabling transaction tracking 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3b08705d 11:22:02,265 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 2 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 ready 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 validated 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: success 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 candidate ready 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - messages: [] 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 prepared 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,266 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 11:22:02,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 11:22:02,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 11:22:02,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:02,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:02,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 submitted to persistence 11:22:02,267 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 11:22:02,268 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 11:22:02,268 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 1 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 11:22:02,269 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 0, nextIndex: 1 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=5413085} is already disabled 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 11:22:02,270 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 11:22:02,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 11:22:02,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 11:22:02,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:02,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 11:22:02,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=183} 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 0 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:22:02,272 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 2, nextIndex: 3 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} has no callback 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:02,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:22:02,274 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$a] 11:22:02,274 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1195788358] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 11:22:02,274 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1195788358] 11:22:02,274 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,274 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 11:22:02,274 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:02,274 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 11:22:02,274 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 129.3 μs 11:22:02,274 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1195788358] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:02,275 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 11:22:02,275 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 11:22:02,276 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 successfully completed direct commit 11:22:02,276 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 2 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:22:02,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1047065572] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 11:22:02,277 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 11:22:02,277 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 11:22:02,277 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1047065572] 11:22:02,277 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,277 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1047065572] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:02,277 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 104.4 μs 11:22:02,277 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,278 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,278 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,278 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 11:22:02,278 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}] 11:22:02,278 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying 1 components for commit 11:22:02,278 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 11:22:02,278 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 directCommit called 11:22:02,278 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: true 11:22:02,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,278 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 canCommit 11:22:02,279 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,279 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2b0ac84a 11:22:02,279 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,279 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 of 1 operations, client version 13 11:22:02,279 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,279 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 ready 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 validated 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: success 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 candidate ready 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - messages: [] 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 prepared 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:22:02,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,281 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 11:22:02,281 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 submitted to persistence 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:22:02,282 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$b] 11:22:02,283 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:02,283 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 11:22:02,284 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 123.3 μs 11:22:02,284 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 successfully completed direct commit 11:22:02,284 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 modifications completed with CommitTransactionReply [version=13] 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 11:22:02,285 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 11:22:02,285 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,286 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 81.30 μs 11:22:02,289 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 11:22:02,289 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,289 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,289 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 11:22:02,289 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,289 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 11:22:02,290 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 11:22:02,290 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,290 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6ec050dc 11:22:02,290 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#983428806] 11:22:02,290 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#983428806] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:02,291 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#983428806, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0, version=13] 11:22:02,291 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:02,291 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#983428806)] 11:22:02,294 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 ReadData operation succeeded 11:22:02,295 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 closeTransaction called 11:22:02,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#983428806)] 11:22:02,295 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,295 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,295 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,295 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 11:22:02,295 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 11:22:02,295 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying 1 components for commit 11:22:02,295 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying transaction for shard cars 11:22:02,295 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 directCommit called 11:22:02,295 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 sending 1 batched modifications, ready: true 11:22:02,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,296 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 canCommit 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6ec050dc 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 of 1 operations, client version 13 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 ready 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 validated 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: success 11:22:02,297 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 candidate ready 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - candidate rootPath: / 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - messages: [] 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: preCommit - successfulFromPrevious: [] 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 prepared 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 11:22:02,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 submitted to persistence 11:22:02,299 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 11:22:02,299 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 11:22:02,299 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:22:02,299 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:22:02,299 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 11:22:02,299 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 4, nextIndex: 5 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: commit - successfulFromPrevious: [] 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:22:02,300 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$e] 11:22:02,301 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,301 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 126.5 μs 11:22:02,301 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 11:22:02,301 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:22:02,301 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 11:22:02,302 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 successfully completed direct commit 11:22:02,302 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 modifications completed with CommitTransactionReply [version=13] 11:22:02,302 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 11:22:02,302 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,302 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 4 11:22:02,302 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,302 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 11:22:02,302 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 11:22:02,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,302 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 11:22:02,302 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 11:22:02,302 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 11:22:02,302 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Queuing TransactionOperation 11:22:02,303 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:22:02,303 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 11:22:02,303 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 11:22:02,303 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,304 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 132.4 μs 11:22:02,305 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 11:22:02,305 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,305 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7f99f9c1 11:22:02,305 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-815282138] 11:22:02,305 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-815282138] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:02,306 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-815282138, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0, version=13] 11:22:02,306 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:02,306 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-815282138)] 11:22:02,308 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 ReadData operation succeeded 11:22:02,308 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 closeTransaction called 11:22:02,308 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-815282138)] 11:22:02,308 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,308 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,308 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,308 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 11:22:02,309 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 11:22:02,309 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying 1 components for commit 11:22:02,309 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying transaction for shard cars 11:22:02,309 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 directCommit called 11:22:02,309 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 sending 1 batched modifications, ready: true 11:22:02,309 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,309 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 canCommit 11:22:02,310 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,310 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7f99f9c1 11:22:02,310 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 of 1 operations, client version 13 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 ready 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 validated 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: success 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 candidate ready 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - candidate rootPath: / 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - messages: [] 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: preCommit - successfulFromPrevious: [] 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 prepared 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 11:22:02,312 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 submitted to persistence 11:22:02,312 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:02,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 5, nextIndex: 6 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: commit - successfulFromPrevious: [] 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$h] 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:22:02,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 11:22:02,314 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,314 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 52.06 μs 11:22:02,315 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 successfully completed direct commit 11:22:02,315 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 modifications completed with CommitTransactionReply [version=13] 11:22:02,315 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 11:22:02,315 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,315 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,315 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 11:22:02,315 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,315 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Queuing TransactionOperation 11:22:02,315 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 5 11:22:02,315 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 11:22:02,315 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 11:22:02,315 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 11:22:02,315 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 11:22:02,316 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:22:02,316 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 11:22:02,316 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 11:22:02,316 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,316 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@f3e544a 11:22:02,316 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 11:22:02,316 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-35902900] 11:22:02,316 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,316 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-35902900] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:02,317 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 109.6 μs 11:22:02,317 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-35902900, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0, version=13] 11:22:02,317 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:02,317 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-35902900)] 11:22:02,319 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 ReadData operation succeeded 11:22:02,319 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 closeTransaction called 11:22:02,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-35902900)] 11:22:02,320 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,320 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,320 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,320 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 11:22:02,320 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 11:22:02,320 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying 1 components for commit 11:22:02,320 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying transaction for shard cars 11:22:02,320 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 directCommit called 11:22:02,320 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 sending 1 batched modifications, ready: true 11:22:02,320 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,320 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 canCommit 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@f3e544a 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 of 1 operations, client version 13 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 ready 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 validated 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: success 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 candidate ready 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - candidate rootPath: / 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - messages: [] 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: preCommit - successfulFromPrevious: [] 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 prepared 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 11:22:02,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 submitted to persistence 11:22:02,323 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 11:22:02,323 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 11:22:02,323 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:22:02,323 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 11:22:02,323 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 11:22:02,323 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: commit - successfulFromPrevious: [] 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from COMMIT_PENDING to COMMITTED 11:22:02,324 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:22:02,325 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$k] 11:22:02,325 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,325 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 11:22:02,325 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:22:02,325 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 11:22:02,325 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 153.1 μs 11:22:02,326 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 successfully completed direct commit 11:22:02,326 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 modifications completed with CommitTransactionReply [version=13] 11:22:02,326 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 11:22:02,326 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 11:22:02,326 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,326 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 11:22:02,326 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,326 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 11:22:02,326 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 11:22:02,326 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 11:22:02,326 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 11:22:02,326 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,326 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Queuing TransactionOperation 11:22:02,326 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:22:02,327 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 11:22:02,327 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 11:22:02,327 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,327 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 102.6 μs 11:22:02,329 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 11:22:02,329 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,329 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@da7e642 11:22:02,330 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#1866525998] 11:22:02,330 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#1866525998] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:02,332 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#1866525998, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0, version=13] 11:22:02,332 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:02,332 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#1866525998)] 11:22:02,335 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 ReadData operation succeeded 11:22:02,335 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 closeTransaction called 11:22:02,335 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#1866525998)] 11:22:02,335 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,335 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,335 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,335 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 11:22:02,335 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 11:22:02,335 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying 1 components for commit 11:22:02,335 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying transaction for shard cars 11:22:02,335 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 directCommit called 11:22:02,335 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 sending 1 batched modifications, ready: true 11:22:02,335 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,336 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 canCommit 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@da7e642 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 of 1 operations, client version 13 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 ready 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 validated 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: success 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 candidate ready 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - candidate rootPath: / 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - messages: [] 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: preCommit - successfulFromPrevious: [] 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 prepared 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 11:22:02,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,338 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 11:22:02,338 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 submitted to persistence 11:22:02,338 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:02,339 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 7, nextIndex: 8 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: commit - successfulFromPrevious: [] 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from COMMIT_PENDING to COMMITTED 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$n] 11:22:02,340 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:22:02,340 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 11:22:02,340 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 114.8 μs 11:22:02,341 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 modifications completed with CommitTransactionReply [version=13] 11:22:02,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 successfully completed direct commit 11:22:02,341 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 11:22:02,341 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 7 11:22:02,341 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,341 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,341 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 11:22:02,341 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 11:22:02,341 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 11:22:02,341 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,341 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 11:22:02,341 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 11:22:02,341 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Queuing TransactionOperation 11:22:02,342 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:22:02,342 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 11:22:02,342 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 11:22:02,342 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:22:02,343 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 132.8 μs 11:22:02,344 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 11:22:02,344 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,344 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@722e5e22 11:22:02,344 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#-1728443462] 11:22:02,344 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#-1728443462] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:02,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#-1728443462, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0, version=13] 11:22:02,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:02,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#-1728443462)] 11:22:02,348 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 ReadData operation succeeded 11:22:02,348 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 closeTransaction called 11:22:02,349 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#-1728443462)] 11:22:02,562 AM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 218, lastApplied : 7, commitIndex : 7 11:22:02,851 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-925146075] 11:22:02,851 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 11:22:02,851 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-899981295] 11:22:02,851 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-899981295] 11:22:02,852 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 11:22:02,852 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:02,852 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:02,852 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 11:22:02,852 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 11:22:02,852 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Queuing TransactionOperation 11:22:02,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 11:22:02,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 11:22:02,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@722e5e22 11:22:02,855 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#698396634] 11:22:02,857 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#698396634, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0, version=13] 11:22:02,857 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:22:02,857 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#698396634)] 11:22:02,860 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 ReadData operation succeeded 11:22:02,860 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 closeTransaction called 11:22:02,860 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#698396634)] 11:22:02,860 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:22:02,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:02,861 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@77893601 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@77893601 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:02,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:02,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 11:22:02,862 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 11:22:02,862 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:02,862 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:02,864 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 7, lastIndex: 7, isVoting: true 11:22:02,864 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 11:22:02,864 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:22:02,864 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 11:22:03,108 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:22:03,279 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 11:22:03,279 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-899981295]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:03,280 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:03,280 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-513: Closing 11:22:03,280 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 11:22:03,280 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:22:03,284 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:22:03,284 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:22:03,284 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:22:03,284 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 11:22:03,284 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:22:03,286 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:03,286 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:22:03.291] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-797)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:22:03.292] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-797)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:22:03.292] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-797)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:22:03.292] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-797)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:03,321 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:22:03,321 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:22:03,321 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 11:22:03,321 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 11:22:03,321 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:22:03,322 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:03,322 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:04,299 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1275311730176089410] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:04,340 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:04,340 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:04,340 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [9216377097809742921] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [02/27/2024 11:22:04.343] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:04,348 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:04,348 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:04,351 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:04.352] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:04,369 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:04,377 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7965740204773360784] 11:22:04,378 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:04,379 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:04,379 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:04,380 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:04,380 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:04,393 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:04,393 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:04,393 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:04,395 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:04,400 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5033734711535946810] 11:22:04,400 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:04,402 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:04,402 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:04,412 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:04,412 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:04,417 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:04,424 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6841919508539139059] 11:22:04,424 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:04,426 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:04,426 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:04,427 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:04,427 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:04,438 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:04,439 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:04,440 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 11:22:04,445 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2000252983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,447 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:04,451 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 11:22:04,451 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:04,451 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@726500f9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4871daaa 11:22:04,451 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5241403 11:22:04,451 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:04,451 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,451 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:04,451 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,451 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709032924451, None)) 11:22:04,452 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032924450,0,0)) 11:22:04,452 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4871daaa to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1217de22 11:22:04,452 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 11:22:04,452 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:04,453 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:04,454 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 11:22:04,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@51c2f5ea 11:22:04,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:04,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,465 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,465 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,465 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] 11:22:04,465 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 11:22:04,465 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,466 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1358834499 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,466 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1358834499] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,466 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 11:22:04,466 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-1540051613] 11:22:04,466 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-1540051613] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:04,468 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:04,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 11:22:04,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 11:22:04,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:22:04,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:22:04,470 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:04,470 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 11:22:04,470 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 11:22:04,470 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 11:22:04,476 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#928069875] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,480 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 11:22:04,480 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:04,480 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@bd8ad85 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e36584e 11:22:04,480 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@661fe7b3 11:22:04,480 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:04,480 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,480 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709032924480, None)) 11:22:04,481 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032924479,0,0)) 11:22:04,481 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e36584e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1d165b3d 11:22:04,481 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 11:22:04,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5944d53e 11:22:04,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:04,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,485 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 11:22:04,485 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 11:22:04,486 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-202194509] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,486 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-115759099 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,486 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-882503009] 11:22:04,486 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-115759099] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,487 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:04,487 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:04,487 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:22:04,487 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:22:04,487 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 11:22:04,487 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 11:22:04,487 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 11:22:04,487 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 11:22:04,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 11:22:04,489 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-882503009] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:04,499 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:04,499 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:04,501 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:04,502 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:04,502 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:04,502 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,502 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,502 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,531 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2000252983] 11:22:04,531 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 11:22:04,532 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] 11:22:04,705 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:04,705 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 11:22:04,705 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:22:04,705 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:04,706 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 11:22:04,706 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:04,706 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:22:04,706 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:22:04,706 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:22:04,706 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 11:22:04,706 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 11:22:05,24 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:05,25 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 11:22:05,25 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,25 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 11:22:05,255 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 11:22:05,255 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,255 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 11:22:05,397 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:05,397 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,400 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,408 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:05,408 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,408 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:05,408 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,409 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,441 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:05,525 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 11:22:05,525 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,525 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 11:22:05,525 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,525 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,527 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,527 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:05,527 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 11:22:05,527 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:05,527 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:05,527 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 11:22:05,528 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:05,528 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,528 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 11:22:05,529 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 11:22:05,529 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:22:05,530 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,531 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 11:22:05,531 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,531 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:22:05,531 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:05,531 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:05,531 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:22:05,531 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 11:22:05,532 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 11:22:05,532 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:05,538 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testRaftCallbackDuringLeadershipDrop commit rateLimit adjusted to 155.8689248333021 11:22:05,538 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,538 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2000252983] 11:22:05,539 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 11:22:05,539 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying 1 components for commit 11:22:05,539 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying transaction for shard cars 11:22:05,539 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 11:22:05,539 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 canCommit 11:22:05,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:05,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5fe12bb] 11:22:05,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5fe12bb] 11:22:05,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387)] for shard cars 11:22:05,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:22:05,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6e8237f5 11:22:05,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 = [ 1 ] 11:22:05,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387)] 11:22:05,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, disabling transaction tracking 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 validated 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: success 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 candidate ready 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - messages: [] 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 prepared 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,540 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 submitted to persistence 11:22:05,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 11:22:05,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 11:22:05,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:05,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 11:22:05,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 11:22:05,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 11:22:05,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:05,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 11:22:05,543 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 11:22:05,543 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,543 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,543 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 11:22:05,543 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=4371310} is already disabled 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 11:22:05,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:22:05,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%231804712387$a] 11:22:05,546 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 successfully completed direct commit 11:22:05,546 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed successfully 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#641975463] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 11:22:05,546 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#641975463] 11:22:05,546 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:22:05,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:05,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 11:22:05,547 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 98.92 μs 11:22:05,547 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#641975463] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,547 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:05,549 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:05,551 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 11:22:05,554 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#427085562] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,554 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:05,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 11:22:05,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:05,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 11:22:05,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 11:22:05,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 11:22:05,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#418131714] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 11:22:05,556 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#418131714] 11:22:05,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 11:22:05,556 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:22:05,556 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 120.1 μs 11:22:05,556 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#418131714] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:05,560 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,560 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:05,560 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@10eb1e92 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fae9744 11:22:05,560 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6c4119ed 11:22:05,560 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709032925560, None)) 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:05,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,560 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032925559,0,0)) 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@62266b4d 11:22:05,561 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fae9744 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5204365c 11:22:05,561 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,561 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#427085562] 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1167630683] 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1167630683] 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1167630683] 11:22:05,562 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 11:22:05,562 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 11:22:05,562 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1167630683] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,562 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#413605511] 11:22:05,563 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#928069875] 11:22:05,563 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1101581993 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,563 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#413605511] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,563 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1101581993] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,563 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,563 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:05,564 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6e8237f5 11:22:05,564 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 11:22:05,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:05,565 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:05,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 11:22:05,565 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 11:22:05,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:22:05,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:22:05,565 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,565 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,565 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 11:22:05,565 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 11:22:05,566 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 11:22:05,566 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2000252983] 11:22:05,566 AM [pool-34-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying 1 components for commit 11:22:05,566 AM [pool-34-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying transaction for shard cars 11:22:05,566 AM [pool-34-thread-1] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 = [ 1 ] 11:22:05,566 AM [pool-34-thread-1] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387)] 11:22:05,566 AM [pool-34-thread-1] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 canCommit 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 validated 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: success 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 candidate ready 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - candidate rootPath: / 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - messages: [] 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 prepared 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,567 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 71 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]]] 11:22:05,568 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 submitted to persistence 11:22:05,570 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 11:22:05,667 AM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] 11:22:05,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387] 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-518: Closing 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 11:22:05,673 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:05,673 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:22:05,925 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:22:05,925 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 11:22:05,926 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 11:22:05,926 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:05,926 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 11:22:05,926 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 11:22:05,926 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 0 - updating term 11:22:05,926 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 11:22:05,927 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:05,927 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,927 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 11:22:05,928 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 11:22:05,928 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 11:22:05,928 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:22:05,929 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:22:05,929 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 11:22:05,932 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:22:05,933 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:05,933 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:05,933 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 11:22:05,933 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:22:05,933 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:22:05,933 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:05,933 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:05,933 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:22:05,937 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 11:22:05,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:22:05,938 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-30427739] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, size=71} 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: commit - successfulFromPrevious: [] 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%231804712387$b] 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 11:22:05,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]] - lastApplied: 3, lastIndex: 3 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:22:05,939 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-30427739] 11:22:05,939 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-521: Closing 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 11:22:05,939 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 11:22:05,939 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 120.7 μs 11:22:05,940 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-30427739] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,940 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed successfully 11:22:05,940 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 successfully completed direct commit 11:22:05,940 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 11:22:05,940 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:22:05,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 11:22:05,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 11:22:05,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 11:22:05,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:22:05,940 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 47.85 μs 11:22:05,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 11:22:05,940 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 11:22:05,940 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 11:22:05,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 11:22:05,943 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:22:05,943 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 23.92 μs 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 11:22:05,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] is present: existingEntryTerm: 4 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] is present: existingEntryTerm: 6 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 2 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:22:05,948 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 11:22:05,948 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:22:05,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 27.94 μs 11:22:05,951 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 11:22:05,951 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 11:22:05,951 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:05,976 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:05,976 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,976 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 11:22:05,976 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 11:22:05,977 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,977 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:22:05,977 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 11:22:05,977 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 11:22:05,977 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:05,977 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:22:05,978 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 11:22:05,978 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:22:05,978 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#1206413732] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1206413732] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:05,978 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:05,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:05,979 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 11:22:05,979 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 11:22:05,979 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 11:22:05,979 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 11:22:05,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:05,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:05,979 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c0cc2a 11:22:05,979 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c0cc2a 11:22:05,979 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 11:22:05,979 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:05,980 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:05,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 11:22:05,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:22:05,982 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 11:22:05,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:05,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:05,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 11:22:05,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 11:22:05,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:05,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:05,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 11:22:06,395 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 11:22:06,395 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1804712387]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:06,395 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:06,395 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-522: Closing 11:22:06,395 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 11:22:06,396 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:06,396 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:22:06,396 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:22:06,396 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:22:06,397 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 11:22:06,397 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:22:06,397 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:06,397 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:06,404 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:06,404 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:06,404 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:06,404 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:06,406 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:06,416 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:22:06,416 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:22:06,416 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:22:06,416 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 11:22:06,416 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:22:06,417 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:06,417 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:07,412 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7965740204773360784] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:07,432 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5033734711535946810] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:07,435 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:07,436 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:07,441 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:07.443] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:07,470 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:07,470 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:07,473 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:07.475] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:07,498 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:07,504 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1569362956021185115] 11:22:07,504 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:07,509 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:07,509 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:07,510 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:07,510 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:07,522 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:07,522 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:07,522 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:07,526 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:07,531 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [9166864184696674530] 11:22:07,532 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:07,533 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:07,533 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:07,539 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:07,539 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:07,547 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:07,559 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8334789563670557461] 11:22:07,559 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:07,561 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:07,561 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:07,570 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:07,570 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:07,572 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:07,573 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:07,575 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 11:22:07,587 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 11:22:07,587 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:22:07,587 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@45a87c88 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5c904ee7 11:22:07,587 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@58afd1ad 11:22:07,587 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709032927587, None)) 11:22:07,591 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-1533287780] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,591 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 11:22:07,594 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:07,595 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032927586,0,0)) 11:22:07,595 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5c904ee7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@afe85b4 11:22:07,596 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 11:22:07,596 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:07,597 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:07,599 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 11:22:07,601 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:07,601 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 11:22:07,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,602 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:07,602 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,602 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@106788e2 11:22:07,602 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:07,602 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,605 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:07,610 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,610 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 11:22:07,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1441974626] 11:22:07,610 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 11:22:07,610 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1441974626] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,610 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1242069113] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,610 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-818890454 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,611 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-818890454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,611 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-262193832] 11:22:07,611 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 11:22:07,611 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-262193832] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:07,612 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:07,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:07,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 11:22:07,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 11:22:07,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 11:22:07,612 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 11:22:07,612 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 11:22:07,613 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:07,614 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 11:22:07,615 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 11:22:07,615 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:22:07,615 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@107a2cb9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3cfb7fb 11:22:07,615 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@654db95e 11:22:07,616 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 11:22:07,616 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 11:22:07,616 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:07,616 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,616 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709032927616, None)) 11:22:07,616 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032927615,0,0)) 11:22:07,617 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3cfb7fb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@70234f49 11:22:07,617 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 11:22:07,622 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6640111f 11:22:07,622 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:07,622 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 11:22:07,622 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,622 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:22:07,622 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-2-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-1679505239] 11:22:07,622 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 11:22:07,623 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-1679505239] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,623 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 11:22:07,623 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#1259501347] 11:22:07,623 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#1607157551] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,623 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#1259501347] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:07,624 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:07,624 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:07,624 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 11:22:07,624 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 11:22:07,624 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:07,624 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 11:22:07,624 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,624 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#1607157551 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 11:22:07,624 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 11:22:07,624 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 11:22:07,624 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 11:22:07,667 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-1533287780] 11:22:07,668 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 11:22:07,668 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1441974626] 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:07,907 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 11:22:07,907 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 11:22:07,908 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 11:22:08,166 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 11:22:08,166 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,166 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 11:22:08,173 AM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:08,456 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 11:22:08,456 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,456 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 11:22:08,527 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:08,527 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:08,528 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:08,528 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 11:22:08,528 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,528 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:08,528 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,528 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,528 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,534 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:08,534 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 11:22:08,534 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,534 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:08,534 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,534 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,587 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:08,747 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 11:22:08,747 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,747 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 11:22:08,747 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:08,750 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 11:22:08,751 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 11:22:08,751 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 11:22:08,754 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 11:22:08,754 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,754 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 11:22:08,754 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 11:22:08,754 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:08,754 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:08,754 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 11:22:08,756 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:22:08,756 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:08,775 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionChainWithSingleShard commit rateLimit adjusted to 155.8689248333021 11:22:08,775 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,776 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1242069113] 11:22:08,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:08,776 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:22:08,776 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 11:22:08,776 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 11:22:08,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 11:22:08,776 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:22:08,776 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 11:22:08,776 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 11:22:08,776 AM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,776 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 11:22:08,776 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 11:22:08,776 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - creating WRITE_ONLY transaction context 11:22:08,776 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:08,776 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 11:22:08,776 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 11:22:08,776 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 11:22:08,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, disabling transaction tracking 11:22:08,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 11:22:08,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 11:22:08,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:08,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:08,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@54fd9ed2 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 ready 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 validated 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: success 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 candidate ready 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - messages: [] 11:22:08,779 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 prepared 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 submitted to persistence 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 11:22:08,780 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 1 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:08,782 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 11:22:08,783 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 0, nextIndex: 1 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=4758340} is already disabled 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 11:22:08,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=173} 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 0 11:22:08,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:08,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 11:22:08,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 11:22:08,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 11:22:08,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 11:22:08,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:22:08,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:22:08,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 11:22:08,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 11:22:08,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 2, nextIndex: 3 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} has no callback 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 11:22:08,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:08,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:22:08,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard$a] 11:22:08,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1211611641] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 11:22:08,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:22:08,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:08,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 11:22:08,791 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1211611641] 11:22:08,791 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:22:08,792 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 94.41 μs 11:22:08,792 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1211611641] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:08,793 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - previous Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 readied - proceeding to FindPrimaryShard 11:22:08,793 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 11:22:08,793 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 11:22:08,793 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 11:22:08,793 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 2 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1315468036] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:22:08,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 11:22:08,794 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1315468036] 11:22:08,794 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:22:08,794 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1315468036] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:08,794 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 78.77 μs 11:22:08,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 11:22:08,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 11:22:08,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2d43b6d1 11:22:08,794 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1228039543] 11:22:08,795 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1228039543] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:08,795 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1228039543, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0, version=13] 11:22:08,795 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:08,795 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1228039543)] 11:22:08,799 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 ReadData operation succeeded 11:22:08,799 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,799 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:08,799 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:08,799 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 11:22:08,799 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 11:22:08,799 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 11:22:08,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 11:22:08,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2d43b6d1 11:22:08,801 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, version=13] 11:22:08,801 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:08,801 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475)] 11:22:08,804 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475] 11:22:08,804 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:08,805 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 11:22:08,805 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:22:08,805 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:22:08,805 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:22:08,805 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: false 11:22:08,805 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475)] 11:22:08,805 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475)] 11:22:08,808 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with BatchedModificationsReply [numBatched=2] 11:22:08,808 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 DataExists operation succeeded 11:22:08,808 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:22:08,808 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475)] 11:22:08,810 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 11:22:08,810 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 11:22:08,810 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 11:22:08,810 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 11:22:08,810 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 successfully completed direct commit 11:22:08,810 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying 1 components for commit 11:22:08,810 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying transaction for shard cars 11:22:08,810 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 directCommit called 11:22:08,810 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: true 11:22:08,810 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#-1062684475)] 11:22:08,811 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 canCommit 11:22:08,812 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, client version 13 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ready 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 validated 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: success 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 candidate ready 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - messages: [] 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 prepared 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 11:22:08,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 submitted to persistence 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=229} 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:08,814 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 11:22:08,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 11:22:08,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 11:22:08,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard_shard-cars-member-2%3Adatastore-testTransactionChainWithSingleShard%400%3A1-2_31%23-1062684475$i] 11:22:08,817 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:22:08,817 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 11:22:08,817 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 11:22:08,817 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 122.5 μs 11:22:08,817 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 11:22:08,818 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 successfully completed direct commit 11:22:08,820 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 11:22:08,820 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:08,820 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:08,820 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 11:22:08,820 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 11:22:08,820 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 11:22:08,821 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 11:22:08,821 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 11:22:08,821 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1753886379] 11:22:08,821 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1753886379] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:08,822 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1753886379, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0, version=13] 11:22:08,822 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:22:08,822 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1753886379)] 11:22:08,823 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 11:22:08,824 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:22:08,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 11:22:08,824 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 11:22:08,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 11:22:08,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 11:22:08,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2dbf1eb3 11:22:08,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2dbf1eb3 11:22:08,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 11:22:08,825 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:08,825 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:22:08,825 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:08,825 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:08,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 11:22:08,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 11:22:08,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 11:22:08,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 11:22:08,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 11:22:08,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 11:22:08,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:22:08,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 11:22:08,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 11:22:08,827 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:22:08,827 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 90.92 μs 11:22:08,827 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:22:08,827 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:08,827 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:08,834 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 3, lastIndex: 3, isVoting: true 11:22:08,834 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 11:22:08,834 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:08,834 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 11:22:09,237 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 11:22:09,237 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1441974626]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:09,237 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 11:22:09,237 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-527: Closing 11:22:09,237 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 11:22:09,238 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 11:22:09,238 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:22:09,238 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:22:09,238 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 11:22:09,238 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:22:09,238 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:22:09,238 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-1533287780] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-1533287780] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:09,239 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1051571377] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1051571377] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:09,239 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:09,239 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:09,244 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:09,245 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:09,245 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:09,245 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:09,247 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:09,260 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:22:09,260 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 11:22:09,260 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 11:22:09,260 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 11:22:09,260 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 11:22:09,261 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:09,261 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:10,256 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1569362956021185115] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:10,276 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:10,276 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:10,278 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:10.280] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,280 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [9166864184696674530] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:10,284 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:10,284 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:10,287 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:10.288] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,302 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:10,308 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1401724175952728188] 11:22:10,308 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:10,309 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:10,309 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:10,311 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:10,311 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:10,324 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:10,324 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:10,324 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:10,324 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:10,329 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7830341999925100437] 11:22:10,329 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:10,331 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,331 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:10,336 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:10,336 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:10,349 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:10,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4422795480548004852] 11:22:10,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:10,362 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:10,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:10,363 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-2023210023)] 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 closeTransaction called 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1228039543)] 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#299889331)] 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 closeTransaction called 11:22:10,394 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-956736640)] [ERROR] [02/27/2024 11:22:10.395] [com.google.common.base.internal.Finalizer] [akka://cluster-test@127.0.0.1:2558/] swallowing exception during message send java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.remote.artery.Association.setupStopQuarantinedTimer(Association.scala:633) at akka.remote.artery.Association.send(Association.scala:399) at akka.remote.artery.ArteryTransport.send(ArteryTransport.scala:707) at akka.remote.RemoteActorRef.$bang(RemoteActorRefProvider.scala:737) at akka.actor.ActorRef.tell(ActorRef.scala:131) at akka.actor.ActorSelection$.rec$1(ActorSelection.scala:287) at akka.actor.ActorSelection$.deliverSelection(ActorSelection.scala:291) at akka.actor.ActorSelection.tell(ActorSelection.scala:50) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.sendOperationAsync(ActorUtils.java:398) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContext.closeTransaction(RemoteTransactionContext.java:84) at org.opendaylight.controller.cluster.datastore.TransactionContextCleanup.finalizeReferent(TransactionContextCleanup.java:60) at jdk.internal.reflect.GeneratedMethodAccessor79.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.base.internal.Finalizer.finalizeReference(Finalizer.java:208) at com.google.common.base.internal.Finalizer.cleanUp(Finalizer.java:181) at com.google.common.base.internal.Finalizer.run(Finalizer.java:146) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:10,396 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 closeTransaction called 11:22:10,396 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1753886379)] 11:22:10,396 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:10,396 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:10,406 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:10,417 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:10,417 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:22:10.424] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,438 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:22:10,438 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:22:10,443 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:10,443 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:10,446 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:10.447] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,462 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:10,466 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3259549840259948388] 11:22:10,467 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:10,468 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:10,468 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:10,478 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:10,478 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:10,478 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:10,478 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:10,479 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:10,482 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:10,489 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-54472833794603163] 11:22:10,489 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:10,490 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,490 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:10,497 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:10,497 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:10,513 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:10,525 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1756011618289501247] 11:22:10,525 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:10,530 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:10,530 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:10,541 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:10,541 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:10,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:10,543 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:10,545 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 11:22:10,553 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:22:10,554 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,564 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 11:22:10,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,565 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:10,565 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,565 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 11:22:10,565 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:22:10,565 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@63b95693 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6706f56e 11:22:10,565 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@63396d56 11:22:10,565 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709032930565, None)) 11:22:10,566 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032930564,0,0)) 11:22:10,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@43a817ba 11:22:10,566 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6706f56e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@738b27c3 11:22:10,566 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 11:22:10,567 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:10,568 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:10,570 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 11:22:10,572 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:10,572 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,572 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,572 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,572 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780] 11:22:10,572 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 11:22:10,573 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 11:22:10,573 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#927737347] 11:22:10,573 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1391027952 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,573 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#927737347] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,574 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:10,574 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:10,574 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 11:22:10,574 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 11:22:10,574 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:10,574 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 11:22:10,574 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 11:22:10,574 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 11:22:10,574 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 11:22:10,592 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1908551965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,593 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:22:10,598 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 11:22:10,598 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:22:10,598 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@35a38921 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@64ce9335 11:22:10,598 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 11:22:10,599 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,600 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1391027952] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,607 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:10,609 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:10,611 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:10,612 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1134c103 11:22:10,612 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709032930598, None)) 11:22:10,612 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032930597,0,0)) 11:22:10,613 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@64ce9335 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@352dce19 11:22:10,613 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 11:22:10,613 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3844e 11:22:10,614 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:10,614 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,614 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 11:22:10,615 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 11:22:10,617 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:10,617 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,617 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,618 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#856658720] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,618 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:10,618 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-2097543919] 11:22:10,623 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-2097543919] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,623 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-606371166 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,623 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-606371166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:10,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:10,624 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:10,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:10,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 11:22:10,625 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 11:22:10,625 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,625 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 11:22:10,625 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 11:22:10,625 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 11:22:10,625 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 11:22:10,664 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] 11:22:10,664 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 11:22:10,664 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780] 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:10,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 11:22:10,869 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 11:22:10,869 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 11:22:11,161 AM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:11,189 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:22:11,189 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,189 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 11:22:11,449 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:22:11,449 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,449 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 11:22:11,490 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:11,490 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:11,491 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:11,491 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,491 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:11,491 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,491 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,515 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:11,516 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,516 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,516 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:11,516 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,516 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,552 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:11,689 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:22:11,689 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,689 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 11:22:11,690 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:11,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 11:22:11,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:11,691 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:11,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 11:22:11,692 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:11,692 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,692 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 11:22:11,696 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:11,696 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 11:22:11,697 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 11:22:11,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 11:22:11,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,699 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 11:22:11,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:11,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:11,699 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 11:22:11,699 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 11:22:11,699 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 11:22:11,699 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:11,771 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1908551965] 11:22:11,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 11:22:11,771 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#856658720] 11:22:11,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,771 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1908551965] 11:22:11,771 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 11:22:11,771 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#856658720] 11:22:11,783 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,783 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,783 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-227-0 of 3 operations, client version 13 11:22:11,787 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-227-0 ready 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 validated 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-227-0: success 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 candidate ready 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: canCommit - candidate rootPath: / 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: canCommit - messages: [] 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: preCommit - successfulFromPrevious: [] 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 prepared 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-227-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 submitted to persistence 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:11,789 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:11,789 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:11,789 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:11,790 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=3399248} is already disabled 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 11:22:11,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:11,791 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: commit - successfulFromPrevious: [] 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from COMMIT_PENDING to COMMITTED 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-762#722065829] 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1705702864] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 11:22:11,792 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1705702864] 11:22:11,792 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:11,792 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:22:11,793 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 145.8 μs 11:22:11,793 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1705702864] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:11,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 11:22:11,794 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 11:22:11,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:11,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 11:22:11,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:22:11,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 11:22:11,794 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] 11:22:11,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-227-0 11:22:11,794 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 11:22:11,795 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#89513243] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 11:22:11,795 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:22:11,795 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:22:11,795 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#89513243] 11:22:11,795 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:11,795 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#89513243] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:11,795 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 108.3 μs 11:22:11,795 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:11,795 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1b31960] 11:22:11,795 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1b31960] 11:22:11,796 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780)] for shard cars 11:22:11,796 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:22:11,796 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6c9817e4 11:22:11,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 11:22:11,798 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,798 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-228-0 of 1 operations, client version 13 11:22:11,798 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,798 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-228-0 ready 11:22:11,799 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 canCommit 11:22:11,799 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780)] resolved 11:22:11,799 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: successfully resolved all cohort actors 11:22:11,799 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 finishCanCommit 11:22:11,799 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-228-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780)] 11:22:11,799 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780)] 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 validated 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-228-0: success 11:22:11,800 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:22:11,800 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit returning result: true 11:22:11,800 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 commit 11:22:11,800 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 finish commit 11:22:11,800 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-228-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780)] 11:22:11,800 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780)] 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 candidate ready 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit - candidate rootPath: / 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit - messages: [] 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: preCommit - successfulFromPrevious: [] 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 prepared 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-228-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 11:22:11,800 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 submitted to persistence 11:22:11,801 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:11,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 11:22:11,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:22:11,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:22:11,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 11:22:11,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:11,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 11:22:11,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:11,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: commit - successfulFromPrevious: [] 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from COMMIT_PENDING to COMMITTED 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23-1785712780$a] 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:11,802 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:22:11,802 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: commit succeeded 11:22:11,803 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:11,803 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 47.10 μs 11:22:11,803 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 11:22:11,803 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:11,803 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6c9817e4 11:22:11,803 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 11:22:11,803 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f8d4745 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5f8d4745 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 11:22:11,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:11,805 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:22:11,805 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:11,805 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-228-0 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:22:11,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:22:11,806 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:11,806 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 44.39 μs 11:22:11,807 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 11:22:11,807 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 11:22:11,807 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 11:22:11,807 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:11,807 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 11:22:11,807 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:22:11,807 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:11,807 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:12,219 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 11:22:12,219 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1785712780]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:12,219 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:12,219 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-532: Closing 11:22:12,219 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:22:12,220 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:22:12,220 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:22:12,220 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:22:12,220 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 11:22:12,220 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:22:12,220 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:12,220 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:12,220 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:22:12,220 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#785628151] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#785628151] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:12,220 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420323262] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:12,221 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#785628151] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#785628151] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:12,221 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:12,221 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:12,227 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:12,227 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:12,227 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:12,227 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:12,229 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:12,243 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:22:12,244 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:22:12,244 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 11:22:12,244 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:22:12,244 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:22:12,245 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:12,245 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:13,231 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [3259549840259948388] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:13,262 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:13,262 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-54472833794603163] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:13,263 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:13,264 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:13.265] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:13,272 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:13,272 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:13,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:13.275] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:13,294 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:13,300 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [124275774479714712] 11:22:13,300 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:13,301 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:13,301 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:13,302 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:13,302 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:13,312 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:13,312 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:13,312 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:13,314 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:13,323 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3775001212414604603] 11:22:13,323 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:13,329 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:13,329 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:13,335 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:13,335 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:13,345 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:13,359 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5213648698824288976] 11:22:13,359 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:13,361 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:13,361 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:13,361 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:13,361 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:13,374 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:13,378 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:13,380 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 11:22:13,387 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,387 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:22:13,394 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:13,398 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 11:22:13,399 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 11:22:13,399 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:22:13,399 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,399 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7d6b2321 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5e73ef1b 11:22:13,399 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:13,399 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,401 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@9e104d0 11:22:13,402 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709032933399, None)) 11:22:13,402 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032933398,0,0)) 11:22:13,403 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5e73ef1b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@bda9f7 11:22:13,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@79ef6925 11:22:13,403 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 11:22:13,403 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:13,405 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:13,408 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:13,408 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 11:22:13,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:13,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,409 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-485490975] 11:22:13,410 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 11:22:13,410 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 11:22:13,411 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-485490975] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,411 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#1503578788 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,411 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#1503578788] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,412 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:13,412 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:13,416 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#1758081634] 11:22:13,417 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 11:22:13,417 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 11:22:13,417 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:22:13,418 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1061872966] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,419 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#1758081634] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,427 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 11:22:13,427 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:22:13,428 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6c22e050 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@c775b72 11:22:13,428 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@22ee9a43 11:22:13,428 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709032933428, None)) 11:22:13,428 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 11:22:13,428 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,428 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:13,428 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,429 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032933427,0,0)) 11:22:13,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4cc815a9 11:22:13,429 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@c775b72 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1b30ca30 11:22:13,429 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 11:22:13,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:13,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:13,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 11:22:13,430 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 11:22:13,430 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 11:22:13,431 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 11:22:13,433 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:13,436 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:13,439 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,439 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,439 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1636338608] 11:22:13,439 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 11:22:13,439 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1636338608] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,440 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-651678220] 11:22:13,440 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#71519689 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,440 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#71519689] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,440 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-651678220] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:13,440 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 11:22:13,441 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:13,441 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:13,441 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 11:22:13,441 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 11:22:13,441 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:13,441 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 11:22:13,441 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,441 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 11:22:13,441 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 11:22:13,441 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 11:22:13,479 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] 11:22:13,480 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 11:22:13,480 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-485490975] 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:13,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 11:22:13,666 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 11:22:13,666 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 11:22:13,924 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 11:22:13,925 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:13,925 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 11:22:14,33 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:14,175 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 11:22:14,175 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,175 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 11:22:14,316 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:14,316 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:14,316 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:14,316 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,316 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,316 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:14,316 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,316 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,317 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:14,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:14,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,378 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:14,414 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 11:22:14,414 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,415 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 11:22:14,415 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,417 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:14,419 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:14,419 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:14,419 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 11:22:14,419 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:14,419 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:14,419 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:22:14,419 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 11:22:14,420 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 11:22:14,420 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,420 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 11:22:14,421 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 11:22:14,421 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:14,421 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:14,421 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 11:22:14,421 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:22:14,421 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:14,423 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 11:22:14,423 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 11:22:14,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:14,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:14,423 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 11:22:14,423 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 11:22:14,435 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 1731439235, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@6fa8c36f} 11:22:14,436 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithSingleShard commit rateLimit adjusted to 74.25352670243916 11:22:14,436 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 11:22:14,436 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1061872966] 11:22:14,436 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:22:14,436 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:14,436 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 11:22:14,436 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 11:22:14,436 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 11:22:14,436 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 11:22:14,436 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 11:22:14,436 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 11:22:14,436 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 11:22:14,436 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] for shard cars 11:22:14,436 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] found - creating WRITE_ONLY transaction context 11:22:14,437 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:14,437 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 11:22:14,437 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 11:22:14,437 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] 11:22:14,438 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, disabling transaction tracking 11:22:14,438 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 11:22:14,438 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 11:22:14,438 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:14,438 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:14,438 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@40e0c66f 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 11:22:14,439 AM [cluster-test-shard-dispatcher-29] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:14,440 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 11:22:14,440 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 11:22:14,444 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:14,444 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:14,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 11:22:14,444 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@1e3aad46 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.SingleCommitCohortProxy$1.onComplete(SingleCommitCohortProxy.java:60) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:14,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 11:22:14,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:14,445 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 11:22:14,445 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 11:22:14,445 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 1 11:22:14,445 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:14,445 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 11:22:14,445 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 11:22:14,446 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 0, nextIndex: 1 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=10742787} is already disabled 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 11:22:14,449 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 2 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 0 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:22:14,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 11:22:14,453 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 11:22:14,453 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 1, nextIndex: 2 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 1 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} has no callback 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:14,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 11:22:14,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 1 11:22:14,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:14,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 1 11:22:14,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 11:22:14,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 11:22:14,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:22:14,454 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 11:22:14,455 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@1e3aad46 is no longer running 11:22:14,455 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:22:14,455 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,455 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 11:22:14,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 11:22:14,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 11:22:14,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68235990 11:22:14,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68235990 11:22:14,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 11:22:14,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:14,455 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:22:14,456 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:14,456 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:14,456 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 11:22:14,456 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:22:14,456 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:14,456 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:14,457 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 11:22:14,457 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 11:22:14,457 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:14,457 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 11:22:14,874 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 11:22:14,874 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-485490975]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:14,875 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,875 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-537: Closing 11:22:14,875 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 11:22:14,875 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:22:14,875 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:22:14,875 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:22:14,875 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 11:22:14,875 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:22:14,876 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:14,876 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1463332087] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1463332087] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:14,876 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:14,876 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:22:14,876 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:14,876 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1463332087] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1463332087] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:14,876 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1432954965] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:14,876 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:14,877 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:14,881 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:14,881 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:14,881 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:14,881 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:14,897 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:22:14,897 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:22:14,897 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 11:22:14,897 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 11:22:14,898 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:22:14,898 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:14,898 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:15,898 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [124275774479714712] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:15,916 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:15,916 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:15,917 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:15,918 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3775001212414604603] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [02/27/2024 11:22:15.919] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:15,925 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:15,925 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:15,927 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:15.928] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:15,945 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:15,952 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2821290484962094866] 11:22:15,952 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:15,953 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:15,953 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:15,958 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:15,959 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:15,963 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:15,963 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:15,964 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:15,966 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:15,970 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1095393462915366374] 11:22:15,970 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:15,972 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:15,972 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:15,979 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:15,979 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:15,987 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:15,995 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7191093884584755667] 11:22:15,995 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:16,3 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:16,3 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:16,5 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:16,5 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:16,13 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:16,14 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:16,15 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 11:22:16,25 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 11:22:16,25 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:22:16,25 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@290db5cc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b33c36f 11:22:16,25 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5188a1d2 11:22:16,25 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:22:16,25 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709032936025, None)) 11:22:16,26 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032936024,0,0)) 11:22:16,26 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b33c36f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@69108531 11:22:16,26 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 11:22:16,26 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1659016854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,27 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:16,28 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:16,30 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@71bf59bd 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,33 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1556274349] 11:22:16,34 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 11:22:16,34 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 11:22:16,35 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1556274349] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,35 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#64727579 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,35 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#64727579] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,35 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1697715357] 11:22:16,35 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1697715357] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,35 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:16,35 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:16,35 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 11:22:16,35 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 11:22:16,35 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 11:22:16,35 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 11:22:16,38 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 11:22:16,38 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 11:22:16,38 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 11:22:16,41 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#2068689738] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,41 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:22:16,45 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:16,52 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:16,54 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:16,56 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:16,61 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 11:22:16,61 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,61 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:16,61 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,62 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 11:22:16,62 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:22:16,62 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6c843c20 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e0590d7 11:22:16,62 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7a3a1bcf 11:22:16,62 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709032936062, None)) 11:22:16,62 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032936061,0,0)) 11:22:16,62 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e0590d7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@e4de250 11:22:16,62 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4e962c0f 11:22:16,62 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 11:22:16,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:16,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,63 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,63 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#274260802] 11:22:16,63 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 11:22:16,64 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 11:22:16,64 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1432468314 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,64 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-303315064] 11:22:16,64 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#274260802] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,64 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1432468314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,64 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-303315064] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:16,65 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:16,65 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:16,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 11:22:16,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 11:22:16,65 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 11:22:16,65 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 11:22:16,65 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 11:22:16,65 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,65 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 11:22:16,65 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 11:22:16,113 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1659016854] 11:22:16,113 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 11:22:16,113 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#1556274349] 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 11:22:16,283 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 11:22:16,283 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 11:22:16,512 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 11:22:16,512 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,512 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 11:22:16,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 11:22:16,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,752 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 11:22:16,974 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:16,974 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:16,974 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:16,974 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,974 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,974 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:16,974 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,974 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,974 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,979 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:16,979 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,979 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,979 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:16,979 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:16,979 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,23 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:17,73 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 11:22:17,73 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,73 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 11:22:17,73 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:17,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 11:22:17,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:17,75 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:17,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 11:22:17,76 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:17,76 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,76 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:17,81 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:17,81 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:17,81 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 11:22:17,82 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:17,82 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:17,82 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 11:22:17,82 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 11:22:17,82 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 11:22:17,82 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 11:22:17,82 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:17,82 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:17,83 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 11:22:17,83 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 11:22:17,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 11:22:17,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,83 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 11:22:17,83 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:17,83 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:17,83 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 11:22:17,83 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 11:22:17,84 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 11:22:17,85 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:17,119 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 11:22:17,119 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#2068689738] 11:22:17,119 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Queuing TransactionOperation 11:22:17,119 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:17,119 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 11:22:17,120 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 11:22:17,120 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars 11:22:17,120 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 11:22:17,120 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, disabling transaction tracking 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 11:22:17,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Creating transaction : member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 11:22:17,123 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#192330116] 11:22:17,123 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#192330116] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:17,123 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Number of entries to be appended = 1 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 11:22:17,124 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#192330116, transactionId=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0, version=13] 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Log size is now 1 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:17,124 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:17,124 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#192330116)] 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 11:22:17,124 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done true 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: 0, nextIndex: 1 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Setting last applied to 0 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=3556002} is already disabled 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:17,125 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Commit index set to 0 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Setting last applied to 0 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 11:22:17,126 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 11:22:17,127 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 ReadData operation succeeded 11:22:17,128 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:22:17,128 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,128 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-542: Closing 11:22:17,129 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 11:22:17,129 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:22:17,129 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:17,129 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:17,135 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:17,135 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:17,135 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:17,135 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:17,137 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:17,145 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 11:22:17,145 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithShardLeaderNotResponding commit rateLimit adjusted to 74.25352670243916 11:22:17,145 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 11:22:17,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:17,145 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:17,145 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 11:22:17,145 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 11:22:17,145 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 11:22:17,146 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying 1 components for commit 11:22:17,146 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying transaction for shard cars 11:22:17,146 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 11:22:17,146 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 canCommit 11:22:17,146 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,146 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:17,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:22:17,244 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:22:17,244 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 11:22:17,244 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 11:22:17,314 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:22:17,344 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:22:17,344 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 11:22:17,344 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 11:22:17,414 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:22:17,444 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:22:17,444 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 11:22:17,444 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 11:22:17,514 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 11:22:17,544 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 11:22:17,544 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 11:22:17,544 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 11:22:17,614 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Creating NoOpTransaction because of error akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:17,614 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:17,614 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.ShardLeaderNotRespondingException: Could not create a READ_WRITE transaction on shard cars. The shard leader isn't responding. at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:213) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.onCreateTransactionComplete(RemoteTransactionContextSupport.java:195) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport$1.onComplete(RemoteTransactionContextSupport.java:136) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:17,615 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:22:17,615 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:17,616 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:22:17,616 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:22:17,616 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:17,616 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:22:17,617 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:22:17,617 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:22:17,617 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 11:22:17,617 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 11:22:17,618 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:22:17,618 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:17,618 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:18,142 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2821290484962094866] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:18,632 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1095393462915366374] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:18,636 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:18,637 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:18,639 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:18.640] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:18,661 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:18,661 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:18,663 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:18.665] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:18,691 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:18,697 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5503235822089884645] 11:22:18,698 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:18,699 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:18,699 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:18,699 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:18,699 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:18,710 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:18,710 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:18,710 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:18,714 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:18,723 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4719095632177165570] 11:22:18,723 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:18,724 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:18,724 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:18,730 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:18,730 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:18,737 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:18,744 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8677003924092004232] 11:22:18,744 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:18,746 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:18,746 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:18,755 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:18,758 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:18,758 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:18,759 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:18,760 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 11:22:18,773 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#65145406] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,773 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 11:22:18,773 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:22:18,773 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@473bd0c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4844d1be 11:22:18,773 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4a86f160 11:22:18,773 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709032938773, None)) 11:22:18,774 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032938772,0,0)) 11:22:18,774 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4844d1be to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3a87a081 11:22:18,774 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 11:22:18,774 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:18,775 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:18,776 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:18,777 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 11:22:18,778 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:18,778 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2c3ca70f 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-186860496] 11:22:18,779 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 11:22:18,780 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 11:22:18,781 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#856662393 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,781 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1578569656] 11:22:18,783 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-186860496] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,783 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#856662393] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,783 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1578569656] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,788 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:18,790 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:18,793 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:18,793 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:18,793 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:18,793 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 11:22:18,793 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 11:22:18,796 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:18,796 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 11:22:18,796 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:22:18,796 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@cffd3aa to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@62c5a9db 11:22:18,796 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@27c72be7 11:22:18,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709032938796, None)) 11:22:18,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032938795,0,0)) 11:22:18,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@62c5a9db to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3594e8ba 11:22:18,797 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 11:22:18,797 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:18,798 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,798 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:22:18,798 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:18,798 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 11:22:18,798 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@60fd7906 11:22:18,798 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:18,798 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,798 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 11:22:18,798 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 11:22:18,801 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:18,801 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,801 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 11:22:18,802 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 11:22:18,802 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:18,802 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1046799393] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,802 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,802 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#1242428032] 11:22:18,802 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#2074412007] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,802 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#1242428032] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:18,803 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:18,803 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:18,803 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:22:18,803 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:18,803 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,803 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 11:22:18,803 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 11:22:18,804 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#2074412007 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:18,804 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 11:22:18,804 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 11:22:18,804 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 11:22:18,848 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#65145406] 11:22:18,848 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 11:22:18,848 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-186860496] 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:22:19,118 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:19,118 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:22:19,119 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:22:19,360 AM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:19,387 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:19,387 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,388 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 11:22:19,678 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:19,678 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,678 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 11:22:19,718 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:19,718 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:19,719 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:19,719 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,719 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,719 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:19,719 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,719 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,719 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,761 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:19,761 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,761 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:19,761 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,775 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:19,997 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:19,997 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,997 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 11:22:19,997 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:19,999 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 11:22:20,1 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 11:22:20,1 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:22:20,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:20,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:22:20,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:20,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:20,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:22:20,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 11:22:20,3 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 11:22:20,3 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:20,6 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleShardTransactionsWithLeaderChanges commit rateLimit adjusted to 74.25352670243916 11:22:20,6 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,6 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1046799393] 11:22:20,6 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 11:22:20,6 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:20,6 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 11:22:20,6 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying 1 components for commit 11:22:20,6 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 11:22:20,6 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying transaction for shard cars 11:22:20,6 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 11:22:20,6 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 canCommit 11:22:20,6 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 11:22:20,6 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] for shard cars 11:22:20,7 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] found - creating WRITE_ONLY transaction context 11:22:20,7 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:20,7 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 11:22:20,7 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 directCommit called 11:22:20,7 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 sending 2 batched modifications, ready: true 11:22:20,7 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, disabling transaction tracking 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 11:22:20,8 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying 2 batched modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Readying Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 of 2 operations, client version 13 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 ready 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 validated 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: success 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 candidate ready 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - messages: [] 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 prepared 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,9 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 submitted to persistence 11:22:20,10 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 11:22:20,10 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:20,10 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 11:22:20,10 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 11:22:20,10 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:20,10 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 11:22:20,10 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 11:22:20,11 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 11:22:20,11 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:20,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 11:22:20,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:20,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 11:22:20,11 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 11:22:20,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 11:22:20,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 11:22:20,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:22:20,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 11:22:20,12 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=3502360} is already disabled 11:22:20,12 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:20,12 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,12 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 11:22:20,12 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 11:22:20,12 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 11:22:20,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 11:22:20,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 11:22:20,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:20,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=181} 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:20,13 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 11:22:20,13 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 11:22:20,13 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:20,13 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 1 11:22:20,13 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 11:22:20,13 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 11:22:20,13 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,14 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:22:20,14 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:20,14 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges$a] 11:22:20,14 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-227629027] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 11:22:20,14 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 11:22:20,14 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-227629027] 11:22:20,14 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:20,14 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 11:22:20,14 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 11:22:20,14 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-227629027] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:20,14 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 148.5 μs 11:22:20,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 1 11:22:20,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:20,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 11:22:20,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 11:22:20,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 11:22:20,15 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 11:22:20,15 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 successfully completed direct commit 11:22:20,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,16 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1755983685] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 11:22:20,16 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1755983685] 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 11:22:20,16 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 11:22:20,16 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1755983685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:20,16 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 122.7 μs 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 11:22:20,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:20,17 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,17 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-547: Closing 11:22:20,17 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:20,17 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 11:22:20,17 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:20,17 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:20,17 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:20,21 AM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:20,21 AM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:20,21 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:20,21 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:20,22 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:20,39 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 11:22:20,89 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1046799393] 11:22:20,89 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 11:22:20,89 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022] 11:22:20,89 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,140 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,190 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,210 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:20,211 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:22:20,211 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 11:22:20,212 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:20,212 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:20,212 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:22:20,247 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:20,253 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-779176467241759752] 11:22:20,253 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:20,256 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:20,256 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:20,256 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:20,256 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:20,263 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:20,264 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:20,265 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 11:22:20,272 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#443093769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:20,273 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:20,279 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:20,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,280 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:22:20,280 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 11:22:20,280 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1d6c708b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e3ba493 11:22:20,280 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4b392b30 11:22:20,281 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709032940281, None)) 11:22:20,281 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032940280,0,0)) 11:22:20,282 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e3ba493 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3d084d06 11:22:20,284 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 11:22:20,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@54ae4fc8 11:22:20,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:20,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,285 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 11:22:20,285 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#969610816] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:20,286 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1994130380 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,286 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-428682931] 11:22:20,286 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1994130380] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:20,286 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 11:22:20,286 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-428682931] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 71 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 181 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 1 to 1 11:22:20,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 11:22:20,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@62d64fa8}, purged=MutableUnsignedLongSet{size=0}} 11:22:20,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:22:20,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 11:22:20,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:20,288 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 586.5 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 11:22:20,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:22:20,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:20,289 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 11:22:20,289 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 11:22:20,289 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 11:22:20,289 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 11:22:20,289 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 11:22:20,335 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1046799393] 11:22:20,335 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 11:22:20,335 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022] 11:22:20,340 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:20,340 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,340 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 11:22:20,340 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,500 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:20,500 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,501 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 11:22:20,501 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,720 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:20,720 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,720 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 11:22:20,720 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:22:20,844 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:22:20,845 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:20,845 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:22:20,845 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:22:20,930 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:20,930 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:20,930 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 11:22:20,931 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,33 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-5503235822089884645] but current is [-779176467241759752]. Reason: ActorSystem terminated 11:22:21,35 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5503235822089884645] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:21,60 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:22:21,60 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,60 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 11:22:21,60 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,68 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#426593169]] intended for system uid [-5503235822089884645], self uid is [-779176467241759752] 11:22:21,68 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] LocalActorRefProvider$$anon$2 - Message [akka.cluster.GossipEnvelope] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [4719095632177165570]. This system with UID [-779176467241759752] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:21,68 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 5 11:22:21,69 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 11:22:21,69 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=5, voteGranted=false] 11:22:21,69 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 5 11:22:21,69 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 11:22:21,69 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 11:22:21,71 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 11:22:21,71 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,71 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 11:22:21,72 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 11:22:21,72 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:22:21,72 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:22:21,72 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 11:22:21,72 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:22:21,72 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 6 - updating term 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=true] 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 7 - updating term 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=true] 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 8 - updating term 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=true] 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 9 - updating term 11:22:21,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 11:22:21,74 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 11:22:21,74 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,74 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=true] 11:22:21,74 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 11:22:21,75 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:21,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=false], my term is 10 11:22:21,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=false], current voteCount: 1 11:22:21,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 10 11:22:21,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 10 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 11:22:21,78 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 11:22:21,78 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:22:21,83 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:21,83 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = null 11:22:21,83 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 11:22:21,83 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 11:22:21,83 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:22:21,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:21,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:21,83 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 11:22:21,84 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:21,89 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,90 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1046799393] 11:22:21,90 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 11:22:21,90 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:21,90 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying 1 components for commit 11:22:21,90 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying transaction for shard cars 11:22:21,90 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 11:22:21,90 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 canCommit 11:22:21,90 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1311783f] 11:22:21,91 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1311783f] 11:22:21,91 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022)] for shard cars 11:22:21,91 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:22:21,91 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4bf8ff64 11:22:21,91 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 = [ 1 ] 11:22:21,91 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022)] 11:22:21,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: handleReadyLocalTransaction for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying local modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 validated 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: success 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 candidate ready 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - candidate rootPath: / 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - messages: [] 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 prepared 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 11:22:21,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 11:22:21,93 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:21,93 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:22:21,93 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,93 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 11:22:21,93 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 submitted to persistence 11:22:21,93 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 11:22:21,95 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - New leader address: akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,95 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,95 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 10 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - New leader address: akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=240} 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 11:22:21,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=10, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=10, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:21,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 11:22:21,97 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 2 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data Simple{size=240} 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: commit - successfulFromPrevious: [] 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges%232004813022$b] 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 10 11:22:21,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:21,98 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 11:22:21,98 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed successfully 11:22:21,98 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 11:22:21,98 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 successfully completed direct commit 11:22:21,98 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 139.0 μs 11:22:21,98 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 11:22:21,98 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#2004813022)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:21,98 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4bf8ff64 11:22:21,98 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 11:22:21,99 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 2 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 2 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data Simple{size=240} 11:22:21,99 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,100 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:22:21,100 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-676986847] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 11:22:21,100 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 10 11:22:21,100 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 11:22:21,100 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 11:22:21,100 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-676986847] 11:22:21,101 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-676986847] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:21,101 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.datastore.ShardDataTreeNotificationPublisherActor$PublishNotifications] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-676986847] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-676986847] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:21,101 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,101 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:22:21,101 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 11:22:21,101 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessagesSuccessful$] from Actor[akka://cluster-test/system/in-memory-journal#1127049905] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#969610816] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#969610816] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:21,101 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessageSuccess] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#969610816] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#969610816] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#969610816] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:21,101 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 11:22:21,101 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:21,101 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:21,101 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:22:21,102 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 11:22:21,102 AM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:21,102 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:21,102 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:22:21,102 AM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:21,102 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:21,102 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:21,107 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:21,107 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:21,113 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:21,113 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:21,113 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:21,113 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:21,115 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:21,137 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:22:21,137 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-82762451] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-82762451] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:21,138 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:22:21,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-550: Closing 11:22:21,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 11:22:21,138 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:22:21,139 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:21,139 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:21,816 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 11:22:21,819 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-5503235822089884645] but current is [-779176467241759752]. Reason: Cluster member removed, previous status [Down] 11:22:22,125 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-779176467241759752] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:22,153 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:22,153 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:22,155 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4719095632177165570] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:22,159 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-5503235822089884645] but current is [-779176467241759752]. Reason: Cluster member removed, previous status [Down] [WARN] [02/27/2024 11:22:22.162] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:22,172 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:22,172 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:22,174 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:22.176] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:22,191 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:22,198 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7223078307095626165] 11:22:22,198 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:22,199 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:22,199 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:22,209 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:22,209 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:22,214 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:22,214 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:22,214 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:22,218 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:22,225 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1034076977853346943] 11:22:22,225 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:22,233 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:22,233 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:22,233 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:22,233 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:22,279 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:22,287 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:22,290 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:22,302 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2200395340765221865] 11:22:22,302 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:22,304 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:22,304 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:22,305 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:22,305 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:22,313 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:22,315 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:22,316 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 11:22:22,319 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 11:22:22,322 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,333 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 11:22:22,333 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:22,334 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2ec8cbea to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20e1e0d9 11:22:22,334 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1786b525 11:22:22,334 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 11:22:22,334 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,334 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:22,334 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,335 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709032942335, None)) 11:22:22,337 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032942334,0,0)) 11:22:22,337 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20e1e0d9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@19fa0408 11:22:22,338 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 11:22:22,338 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:22,339 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:22,341 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@68e7e96f 11:22:22,341 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:22,341 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,341 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 11:22:22,343 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,343 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,343 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:22,344 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 11:22:22,344 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 11:22:22,344 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,344 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-1303521940] 11:22:22,344 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-571628133 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,345 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-571628133] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,345 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-1303521940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,345 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:22,345 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:22,345 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:22,345 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:22:22,345 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 11:22:22,345 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 11:22:22,345 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 11:22:22,346 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 11:22:22,348 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#418675229] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,349 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 11:22:22,349 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:22,351 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 11:22:22,352 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:22,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 11:22:22,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:22,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:22,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,355 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 11:22:22,355 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:22,355 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4961c2ba to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9119d2d 11:22:22,355 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@71f2c780 11:22:22,355 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709032942355, None)) 11:22:22,355 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032942354,0,0)) 11:22:22,356 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5be63b95 11:22:22,356 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9119d2d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@121551d6 11:22:22,356 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 11:22:22,356 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:22,356 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:22,357 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,357 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:22,357 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1628614872] 11:22:22,357 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 11:22:22,357 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1628614872] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,358 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 11:22:22,358 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#878440850] 11:22:22,358 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#762042295 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:22,358 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#762042295] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,358 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#878440850] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:22,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:22,359 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:22,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:22,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:22:22,359 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 11:22:22,359 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 11:22:22,359 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 11:22:22,359 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,359 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 11:22:22,359 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 11:22:22,406 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] 11:22:22,406 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 11:22:22,406 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:22,639 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:22,639 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 11:22:22,639 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:22:22,639 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,639 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 11:22:22,639 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:22,639 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 11:22:22,640 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:22,640 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:22:22,640 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 11:22:22,640 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 11:22:22,949 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 11:22:22,949 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:22,949 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 11:22:22,963 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:23,199 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 11:22:23,199 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:23,199 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 11:22:23,220 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:23,220 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:23,220 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:23,220 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:22:23,220 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:23,220 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:23,220 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,220 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:23,220 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,228 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:23,229 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:22:23,229 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,229 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:23,229 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,229 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,321 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:23,508 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 11:22:23,508 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:23,508 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 11:22:23,509 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:23,512 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:23,515 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:23,515 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:23,515 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 11:22:23,516 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 11:22:23,516 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:22:23,516 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithIsolatedLeader commit rateLimit adjusted to 74.25352670243916 11:22:23,516 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,516 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] 11:22:23,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:23,517 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Queuing TransactionOperation 11:22:23,517 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 11:22:23,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f34adf2] 11:22:23,517 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] 11:22:23,517 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Queuing TransactionOperation 11:22:23,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:23,517 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f34adf2] 11:22:23,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f34adf2] 11:22:23,517 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] 11:22:23,517 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Queuing TransactionOperation 11:22:23,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:23,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f34adf2] 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059)] for shard cars 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@53ace2c2 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f34adf2] 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059)] for shard cars 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@53ace2c2 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f34adf2] 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059)] for shard cars 11:22:23,517 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@53ace2c2 11:22:23,519 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 11:22:23,519 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:23,519 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 11:22:23,519 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:23,519 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:23,519 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:22:23,519 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 11:22:23,521 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 11:22:23,521 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:23,527 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:23,527 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,527 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 11:22:23,528 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 11:22:23,528 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,528 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 11:22:23,528 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 11:22:23,528 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 11:22:23,528 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 11:22:23,528 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:22:23,528 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying 1 components for commit 11:22:23,528 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying transaction for shard cars 11:22:23,528 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 = [ 1 ] 11:22:23,529 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059)] 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, disabling transaction tracking 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:23,529 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}]]] 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 validated 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: success 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,529 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 candidate ready 11:22:23,529 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - messages: [] 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 prepared 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,530 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:23,530 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:22:23,530 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#418675229] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#418675229] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 11:22:23,530 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-751228499] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-751228499] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:23,530 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#418675229] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#418675229] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,530 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 submitted to persistence 11:22:23,531 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:23,531 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 11:22:23,531 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:23,531 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 11:22:23,531 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:23,532 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 11:22:23,532 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:22:23,533 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:23,582 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:23,628 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:23,633 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:23,683 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:23,734 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:23,749 AM [cluster-test-shard-dispatcher-16] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 11:22:23,751 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:22:23,751 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:23,751 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 11:22:23,751 AM [cluster-test-shard-dispatcher-16] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-555: Closing 11:22:23,751 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:22:23,751 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 11:22:23,753 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 11:22:23,753 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:23,753 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:23,785 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059] 11:22:23,785 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying 1 components for commit 11:22:23,785 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying transaction for shard cars 11:22:23,785 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 = [ 1 ] 11:22:23,785 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059)] 11:22:23,786 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 11:22:23,786 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@34433bd4 to retry 11:22:23,786 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 canCommit 11:22:24,199 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Timer expired for message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@34433bd4 11:22:24,200 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] LocalThreePhaseCommitCohort - Failed to prepare transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 on backend org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Could not process ready local transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0. Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport$MessageInfo.timedOut(ShardTransactionMessageRetrySupport.java:102) at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport.onTimerMessage(ShardTransactionMessageRetrySupport.java:72) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:410) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:24,201 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 11:22:24,201 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying 1 components for commit 11:22:24,201 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying transaction for shard cars 11:22:24,201 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 = [ 1 ] 11:22:24,201 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1033775059)] 11:22:24,201 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:24,201 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,201 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:24,201 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@1b0acfa9 to retry 11:22:24,201 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:22:24,204 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:24,206 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 11:22:24,207 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 11:22:24,208 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:24,208 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 11:22:24,208 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 11:22:24,209 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@25b064e7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@59e6f5a7 11:22:24,209 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,209 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@174eb056 11:22:24,209 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:24,209 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:22:24,209 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:24,209 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:24,209 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:24,209 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,209 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709032944209, None)) 11:22:24,209 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032944208,0,0)) 11:22:24,210 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@34980472 11:22:24,210 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@59e6f5a7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1925cccb 11:22:24,210 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 11:22:24,210 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 canCommit 11:22:24,210 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:24,210 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1926408521] 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1926408521] 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1926408521] 11:22:24,211 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 11:22:24,212 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 11:22:24,212 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#1458489192 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,212 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#1119815078] 11:22:24,212 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:22:24,212 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 11:22:24,212 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:22:24,212 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:24,212 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:24,212 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:24,213 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:24,213 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:22:24,213 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 11:22:24,213 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:24,213 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 11:22:24,213 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 11:22:24,213 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 11:22:24,213 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 11:22:24,228 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,229 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 11:22:24,231 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=170} 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:24,232 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 1, nextIndex: 2 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 0 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:24,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=704783967} is already disabled 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 1 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-1033775059$a] 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-632353179] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 11:22:24,234 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 successfully completed direct commit 11:22:24,234 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed successfully 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 11:22:24,234 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 canCommit 11:22:24,234 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-632353179] 11:22:24,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 11:22:24,235 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-556: Closing 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Retrying message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@1b0acfa9 11:22:24,235 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,235 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 253.5 μs 11:22:24,235 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-632353179] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 validated 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: success 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 candidate ready 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - candidate rootPath: / 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - messages: [] 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 prepared 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,236 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 68 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 11:22:24,237 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 11:22:24,237 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 submitted to persistence 11:22:24,237 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:22:24,237 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 1 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 11:22:24,238 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1500754316] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=68} 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 11:22:24,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:24,239 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1500754316] 11:22:24,240 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:22:24,240 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 11:22:24,240 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 11:22:24,240 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 11:22:24,240 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 143.4 μs 11:22:24,242 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 2, nextIndex: 3 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: commit - successfulFromPrevious: [] 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 11:22:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-1033775059$b] 11:22:24,245 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed successfully 11:22:24,245 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:22:24,245 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 successfully completed direct commit 11:22:24,246 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 11:22:24,246 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:24,246 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 21.81 μs 11:22:24,246 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:24,246 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 11:22:24,246 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:24,246 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 11:22:24,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 11:22:24,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 11:22:24,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@34a024e4 11:22:24,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@34a024e4 11:22:24,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 11:22:24,246 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 11:22:24,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:24,247 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:24,247 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 2 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 11:22:24,247 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 11:22:24,247 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:22:24,248 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 20.77 μs 11:22:24,248 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 11:22:24,248 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 11:22:24,249 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:24,249 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:24,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 2, lastIndex: 2, isVoting: true 11:22:24,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 11:22:24,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:24,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 11:22:24,250 AM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:26,269 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.023 s 11:22:26,269 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 11:22:26,270 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 11:22:26,270 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-559: Closing 11:22:26,270 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 11:22:26,270 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 11:22:26,270 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 11:22:26,270 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:22:26,271 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:22:26,271 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:22:26,272 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 11:22:26,272 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:22:26,272 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:26,272 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:26,272 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:26,272 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-89639548] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:26,273 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:26,273 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:26,279 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:26,279 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:26,279 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:26,279 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:26,285 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:26,300 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:22:26,301 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:22:26,301 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 11:22:26,301 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 11:22:26,301 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 11:22:26,302 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:26,302 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:27,290 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7223078307095626165] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:27,316 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:27,316 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:27,318 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:27.319] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:27,326 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1034076977853346943] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:27,335 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:27,335 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:27,338 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:27.339] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:27,358 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:27,365 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4527351926053318484] 11:22:27,365 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:27,367 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:27,367 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:27,368 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:27,368 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:27,382 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:27,382 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:27,382 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:27,383 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:27,394 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7990270804959386384] 11:22:27,394 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:27,400 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:27,400 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:27,402 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:27,402 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:27,411 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 closeTransaction called 11:22:27,411 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#192330116)] 11:22:27,419 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:27,425 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7110029357395076605] 11:22:27,426 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:27,427 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:27,427 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:27,436 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:27,437 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:27,438 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:27,438 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:27,439 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 11:22:27,446 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-689142062] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:22:27,453 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:22:27,453 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:27,453 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5a0d53af to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@315431e1 11:22:27,453 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7456a9af 11:22:27,453 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 11:22:27,453 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709032947453, None)) 11:22:27,453 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,454 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032947452,0,0)) 11:22:27,454 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,454 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@315431e1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@71d889cc 11:22:27,454 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:27,454 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,454 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,454 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 11:22:27,454 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@60faba0c 11:22:27,455 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984] 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,456 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189] 11:22:27,457 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 11:22:27,457 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 11:22:27,457 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:27,457 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 11:22:27,458 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1643288762] 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1622848888] 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1855344698 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-2054257678 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-2054257678] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1643288762] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1855344698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,459 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1622848888] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,460 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 11:22:27,461 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:27,461 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:27,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 11:22:27,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 11:22:27,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:27,462 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:27,463 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 11:22:27,463 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 11:22:27,463 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:27,463 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 11:22:27,464 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:22:27,464 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:22:27,464 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 11:22:27,466 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:27,466 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 11:22:27,466 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:22:27,466 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:22:27,466 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 11:22:27,471 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1207096949] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,477 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:27,481 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:27,487 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:22:27,503 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:27,506 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:27,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 11:22:27,510 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:22:27,510 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:27,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,510 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@711572ab to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@d5bb71c 11:22:27,510 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@485938d3 11:22:27,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:27,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:27,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,511 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709032947510, None)) 11:22:27,511 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032947509,0,0)) 11:22:27,511 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@d5bb71c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@14b44a8c 11:22:27,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@331f40ee 11:22:27,511 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 11:22:27,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:27,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,512 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-1051102790] 11:22:27,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:27,513 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 11:22:27,513 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-1051102790] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,513 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1564426604] 11:22:27,513 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-625491629 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,513 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 11:22:27,513 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-625491629] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,513 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1564426604] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,513 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,513 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:27,513 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#756257208] 11:22:27,513 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 11:22:27,514 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#756257208] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,514 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#1842257031 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:27,514 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#1842257031] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,514 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#54007402] 11:22:27,514 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#54007402] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:27,514 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 11:22:27,514 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,514 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:22:27,514 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:27,514 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 11:22:27,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 11:22:27,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 11:22:27,515 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,515 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:22:27,515 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:22:27,515 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 11:22:27,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-689142062] 11:22:27,562 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 11:22:27,562 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984] 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:27,706 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 11:22:27,706 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:22:27,707 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 11:22:27,726 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:22:27,726 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 11:22:27,966 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:22:27,966 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:27,966 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:22:28,32 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:28,36 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:22:28,36 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,36 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:22:28,112 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:28,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:22:28,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:28,256 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:22:28,315 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:22:28,316 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,316 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,388 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,389 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:28,389 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,400 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,447 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:28,496 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:22:28,496 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:28,497 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:22:28,497 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:28,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:28,500 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:28,500 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:28,500 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 11:22:28,500 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:28,500 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:28,500 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:22:28,500 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:22:28,501 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:22:28,501 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:22:28,501 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:28,501 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:28,501 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 11:22:28,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 11:22:28,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:28,501 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 11:22:28,506 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:22:28,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:28,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:28,506 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:22:28,506 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:28,616 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:22:28,616 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,616 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:22:28,616 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:28,618 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,618 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:28,619 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:28,619 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:22:28,620 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 11:22:28,620 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 11:22:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 11:22:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,621 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:22:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:28,621 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 11:22:28,621 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 11:22:28,621 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:22:28,622 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:28,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-689142062] 11:22:28,718 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 11:22:28,718 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189] 11:22:28,719 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithMultipleShards commit rateLimit adjusted to 74.25352670243916 11:22:28,719 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1207096949] 11:22:28,719 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:22:28,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1207096949] 11:22:28,720 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:22:28,720 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 11:22:28,720 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 11:22:28,720 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:22:28,720 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 11:22:28,720 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 11:22:28,720 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 11:22:28,720 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:28,720 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:22:28,720 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:22:28,720 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 11:22:28,720 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 11:22:28,721 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:22:28,721 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 11:22:28,721 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 11:22:28,721 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 11:22:28,721 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 11:22:28,722 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 11:22:28,722 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,722 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 11:22:28,723 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,723 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,723 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,723 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,723 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:28,723 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,723 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 11:22:28,724 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 11:22:28,724 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,724 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 11:22:28,724 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,724 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 11:22:28,725 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,725 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984)] resolved 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189)] resolved 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984)] 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984)] 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:22:28,726 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 11:22:28,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:28,726 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,727 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 11:22:28,727 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=9751754} is already disabled 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=9122120} is already disabled 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:28,732 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:28,732 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 11:22:28,733 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:22:28,733 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:28,733 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:22:28,733 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:22:28,733 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 11:22:28,733 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 11:22:28,734 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:22:28,734 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:22:28,734 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189)] 11:22:28,734 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189)] 11:22:28,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 11:22:28,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:28,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 11:22:28,736 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:22:28,736 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 11:22:28,736 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 11:22:28,736 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 11:22:28,736 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984)] 11:22:28,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984)] 11:22:28,736 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189)] 11:22:28,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#1075002189)] 11:22:28,738 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,738 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 11:22:28,738 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 11:22:28,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 175 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 11:22:28,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 11:22:28,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 11:22:28,740 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:22:28,740 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=175} 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:22:28,741 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:28,742 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 11:22:28,742 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:22:28,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-people-testWriteTransactionWithMultipleShards%231075002189$f] 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:28,743 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-cars-testWriteTransactionWithMultipleShards%23-144278984$e] 11:22:28,744 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1539024484] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 11:22:28,744 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#2085472014] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#2085472014] 11:22:28,744 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1539024484] 11:22:28,744 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:28,744 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 135.3 μs 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#2085472014] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1539024484] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:28,744 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 87.72 μs 11:22:28,745 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:28,745 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:28,745 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:22:28,746 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 11:22:28,746 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,746 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 11:22:28,746 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:28,746 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - trying create transaction 11:22:28,746 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 11:22:28,746 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1218603193] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 11:22:28,746 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1218603193] 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:28,746 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:28,746 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 49.56 μs 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 11:22:28,746 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:22:28,747 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1218603193] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:28,747 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#2142948430] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 11:22:28,747 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#2142948430] 11:22:28,747 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:28,747 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:28,747 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:22:28,747 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 46.06 μs 11:22:28,747 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#2142948430] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:28,747 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:22:28,748 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#222396366] 11:22:28,748 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#222396366] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:28,754 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#222396366, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 11:22:28,754 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 11:22:28,754 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#222396366)] 11:22:28,756 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 11:22:28,756 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 11:22:28,756 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 11:22:28,756 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - trying create transaction 11:22:28,756 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 11:22:28,756 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 11:22:28,757 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:22:28,757 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1017014274] 11:22:28,757 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1017014274, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 11:22:28,757 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 11:22:28,757 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1017014274)] 11:22:28,759 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 11:22:28,759 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:28,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 11:22:28,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 11:22:28,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 11:22:28,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 11:22:28,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@46798de5 11:22:28,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@547b50f0 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:28,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@46798de5 11:22:28,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@547b50f0 11:22:28,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:22:28,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:28,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:28,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:28,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:28,760 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 11:22:28,760 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 11:22:28,760 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:28,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:28,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:28,761 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:28,761 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:28,761 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:28,763 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:22:28,763 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:22:28,763 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:28,763 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:22:28,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:22:28,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:22:28,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:28,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:22:28,980 AM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 215, lastApplied : 1, commitIndex : 1 11:22:28,980 AM [cluster-test-shard-dispatcher-30] [WARN] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 215, lastApplied : 1, commitIndex : 1 11:22:29,176 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:22:29,176 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-144278984]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:29,176 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 11:22:29,176 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 11:22:29,176 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-567: Closing 11:22:29,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-566: Closing 11:22:29,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:22:29,176 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:22:29,177 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:22:29,177 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:22:29,177 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:22:29,177 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:22:29,178 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:22:29,178 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:29,178 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:29,187 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:29,187 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:29,187 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:29,187 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:29,199 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:22:29,199 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 11:22:29,199 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 11:22:29,199 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 11:22:29,199 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 11:22:29,199 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:22:29,200 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:22:29,200 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:22:29,200 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:29,200 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:30,198 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4527351926053318484] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:30,214 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:30,214 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:30,216 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:30.217] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:30,220 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7990270804959386384] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:30,223 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:30,223 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:30,225 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:30.226] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:30,248 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:30,261 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8219749090283494651] 11:22:30,261 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:30,269 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:30,269 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:30,269 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:30,269 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:30,274 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:30,274 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:30,274 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:30,277 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:30,283 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1869553224039645473] 11:22:30,283 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:30,284 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:30,284 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:30,286 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:30,286 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:30,300 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:30,311 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7296018058070154314] 11:22:30,311 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:30,313 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:30,313 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:30,313 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:30,313 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:30,326 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:30,327 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:30,329 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 11:22:30,333 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:30,337 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 11:22:30,341 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:30,344 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-897102262] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,345 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 11:22:30,345 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:22:30,345 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@74eb3e16 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43524075 11:22:30,345 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@a1e3049 11:22:30,346 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 11:22:30,346 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,346 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,346 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:30,346 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,346 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,346 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709032950346, None)) 11:22:30,346 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032950345,0,0)) 11:22:30,346 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43524075 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7f199645 11:22:30,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@54cf5dd3 11:22:30,347 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 11:22:30,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:30,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,347 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:30,347 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715] 11:22:30,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,348 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,348 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,348 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547] 11:22:30,348 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 11:22:30,348 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 11:22:30,349 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 11:22:30,349 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#281621739 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-500402520] 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#281621739] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-500402520] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#792119696 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,349 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 11:22:30,349 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#792119696] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,350 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 11:22:30,352 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:30,352 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:30,352 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:30,352 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:30,352 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 11:22:30,352 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:30,352 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 11:22:30,352 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 11:22:30,352 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:30,352 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 11:22:30,352 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 11:22:30,352 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:22:30,352 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 11:22:30,358 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1696933098] 11:22:30,358 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1696933098] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,358 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 11:22:30,358 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 11:22:30,358 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 11:22:30,358 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:22:30,358 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 11:22:30,362 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 11:22:30,362 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2094654634] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,363 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 11:22:30,363 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:22:30,363 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4efcfb00 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4387b6cc 11:22:30,363 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@68530d8 11:22:30,363 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709032950363, None)) 11:22:30,364 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032950362,0,0)) 11:22:30,364 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4387b6cc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@77c49034 11:22:30,364 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4386cb14 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:30,365 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:30,366 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,366 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:30,366 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1803435128] 11:22:30,366 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 11:22:30,366 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 11:22:30,366 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 11:22:30,367 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 11:22:30,367 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 11:22:30,367 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,367 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:22:30,367 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#2117880418] 11:22:30,367 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1137045080 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 11:22:30,367 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-513332891] 11:22:30,367 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-614820392 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:30,367 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#520293274] 11:22:30,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1803435128] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#2117880418] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1137045080] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-513332891] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-614820392] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#520293274] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:30,369 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:30,369 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 11:22:30,369 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:22:30,369 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 11:22:30,369 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 11:22:30,369 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 11:22:30,372 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:30,372 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 11:22:30,372 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,372 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 11:22:30,372 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 11:22:30,372 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 11:22:30,375 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:30,381 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:30,415 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-897102262] 11:22:30,415 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 11:22:30,415 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715] 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 11:22:30,593 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 11:22:30,593 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 11:22:30,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:30,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:22:30,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 11:22:30,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,643 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 11:22:30,644 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:30,644 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 11:22:30,644 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:22:30,644 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 11:22:30,644 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 11:22:30,644 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 11:22:30,843 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:22:30,843 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:22:30,843 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 11:22:30,896 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:30,906 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:30,963 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:22:30,963 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:30,963 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 11:22:31,133 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:22:31,133 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:22:31,133 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 11:22:31,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:22:31,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:31,263 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 11:22:31,274 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:31,274 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,274 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,281 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:31,281 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 11:22:31,281 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,281 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 11:22:31,281 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,281 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:31,282 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,282 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,282 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,282 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,328 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:31,383 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:22:31,383 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:22:31,383 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 11:22:31,383 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,384 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:31,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 11:22:31,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:31,385 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:31,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 11:22:31,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:31,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 11:22:31,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:31,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:31,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:31,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 11:22:31,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:31,387 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:31,387 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:22:31,387 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 11:22:31,387 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 11:22:31,387 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:22:31,387 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:31,387 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:31,387 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 11:22:31,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 11:22:31,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 11:22:31,388 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:22:31,388 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:31,388 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:31,388 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 11:22:31,389 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:22:31,389 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:31,505 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 11:22:31,533 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 11:22:31,533 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:31,533 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 11:22:31,533 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,534 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:31,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 11:22:31,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:31,535 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:31,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 11:22:31,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:31,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:31,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:31,536 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:31,536 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:31,536 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 11:22:31,536 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:31,536 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:31,536 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:22:31,536 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 11:22:31,537 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 11:22:31,537 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:22:31,537 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:31,537 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:31,537 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 11:22:31,537 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 11:22:31,538 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 11:22:31,538 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:31,538 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 11:22:31,538 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 11:22:31,538 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:31,538 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:31,538 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 11:22:31,538 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:22:31,538 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:31,621 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-897102262] 11:22:31,621 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 11:22:31,621 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547] 11:22:31,622 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 11:22:31,622 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 changed state from IDLE to TX_OPEN 11:22:31,624 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 11:22:31,624 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:31,624 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:31,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2094654634] 11:22:31,624 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:31,625 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 11:22:31,625 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 11:22:31,625 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:22:31,625 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 11:22:31,625 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:22:31,625 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:22:31,625 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 11:22:31,625 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:22:31,625 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:22:31,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2094654634] 11:22:31,625 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 11:22:31,625 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:22:31,625 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 11:22:31,625 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 11:22:31,625 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:22:31,625 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 11:22:31,625 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 11:22:31,626 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5831dade, attempting to connect 11:22:31,626 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@26161666, attempting to connect 11:22:31,626 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@7fcbbf75 11:22:31,626 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@2d31d712 11:22:31,626 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 changed state from TX_OPEN to IDLE 11:22:31,626 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:22:31,626 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 11:22:31,626 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@a6be3ec 11:22:31,626 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 11:22:31,626 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@32a89d72 11:22:31,643 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 11:22:31,645 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=30031, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:22:31,645 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 11:22:31,645 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=8981, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:22:31,650 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:22:31,650 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:22:31,650 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:22:31,650 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,650 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1f1d651a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 11:22:31,650 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:22:31,650 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:22:31,650 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=804959955305} 11:22:31,650 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 11:22:31,651 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,651 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 11:22:31,651 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 11:22:31,651 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:22:31,651 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2224/0x00000008016a8238@3be1dc1 11:22:31,651 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY}, enqueuedTicks=804961630584} 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY}} 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 410 bytes, total slices to send: 1 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@31b4d20f 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=804961826258} 11:22:31,652 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}} 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 290 bytes, total slices to send: 1 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@33dc62a9 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@688dc783 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3865b396, splicing queue 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@688dc783 queue spliced 0 messages 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 2.908 ms 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@10d75564 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:22:31,653 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=804960708113} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2224/0x00000008016a8238@101e8686 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY}, enqueuedTicks=804961674705} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY}} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 418 bytes, total slices to send: 1 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@c82f8df 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=804961865139} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 291 bytes, total slices to send: 1 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4698ee54 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5ed6dea9 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@279f9479, splicing queue 11:22:31,654 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5ed6dea9 queue spliced 0 messages 11:22:31,655 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 1.349 ms 11:22:31,657 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,657 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,657 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:22:31,657 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3c1c85bc 11:22:31,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 11:22:31,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@ece5baf 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 ready 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 11:22:31,658 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 to ready 11:22:31,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,659 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 is already in state READY (READY) 11:22:31,659 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating canCommit 11:22:31,659 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 ready 11:22:31,659 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 validated 11:22:31,659 AM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:31,659 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed canCommit 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 validated 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:31,659 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 11:22:31,660 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,660 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,665 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 11:22:31,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:22:31,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:31,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 11:22:31,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:22:31,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 11:22:31,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2226/0x00000008016a8a48@3afa7093 11:22:31,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4538131a}, purged=MutableUnsignedLongSet{size=0}} 11:22:31,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:31,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY}, enqueuedTicks=804961674705, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=804961865139, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY}, enqueuedTicks=804961630584, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=804961826258, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 11:22:31,670 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=804959955305, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 11:22:31,670 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@33107007 11:22:31,670 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 11:22:31,670 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6e54849e}, purged=MutableUnsignedLongSet{size=0}} 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,671 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:22:31,671 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 11:22:31,671 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@1084c3b3 11:22:31,671 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,671 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805006597477} 11:22:31,671 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 11:22:31,671 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@55199d62 11:22:31,671 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 11:22:31,671 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 11:22:31,671 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@65b94f8e 11:22:31,671 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805006847443} 11:22:31,675 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating preCommit 11:22:31,676 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating preCommit 11:22:31,676 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,676 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,677 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 candidate ready 11:22:31,677 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 candidate ready 11:22:31,677 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 11:22:31,677 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - candidate rootPath: / 11:22:31,677 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - messages: [] 11:22:31,677 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 11:22:31,677 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - messages: [] 11:22:31,677 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 prepared 11:22:31,677 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: preCommit - successfulFromPrevious: [] 11:22:31,677 AM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:31,677 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 prepared 11:22:31,677 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:31,677 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed preCommit 11:22:31,677 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed preCommit 11:22:31,679 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 11:22:31,679 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805006597477, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 11:22:31,679 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:22:31,679 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 11:22:31,679 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805006847443, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 11:22:31,680 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@39c5f8f7 11:22:31,680 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 3 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@39c5f8f7 11:22:31,680 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,680 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805015298237} 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@20ae8020 11:22:31,680 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 3 11:22:31,680 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@20ae8020 11:22:31,680 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805015549443} 11:22:31,682 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating commit 11:22:31,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating commit 11:22:31,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 177 11:22:31,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 11:22:31,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 11:22:31,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:31,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 11:22:31,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:31,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:31,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:31,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:31,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:31,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 11:22:31,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 11:22:31,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 submitted to persistence 11:22:31,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 submitted to persistence 11:22:31,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:31,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=181} 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=177} 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:22:31,685 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,686 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,686 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 11:22:31,686 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2226/0x00000008016a8a48@6a83729a 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@34d94ece}, purged=MutableUnsignedLongSet{size=0}} 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1045590508] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: commit - successfulFromPrevious: [] 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:31,687 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1507259356] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 11:22:31,687 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1045590508] 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:31,687 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,687 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:31,688 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 72.01 μs 11:22:31,688 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1045590508] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:31,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1507259356] 11:22:31,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,688 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1507259356] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:31,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 90.36 μs 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805015298237, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} doCommit completed 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1f1d651a completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 4 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@4a5614f 11:22:31,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 4 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6d856047 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805029663737} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=804960708113, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805015549443, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} doCommit completed 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@10d75564 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 4 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@19e2ad48 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 4 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6af18378 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:31,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805030485836} 11:22:31,695 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 11:22:31,695 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:31,695 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 11:22:31,695 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:22:31,695 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#995298239] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@bee8bab}, purged=MutableUnsignedLongSet{size=0}} 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 11:22:31,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1960978685] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 11:22:31,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:22:31,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:22:31,697 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#995298239] 11:22:31,697 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,697 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#995298239] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:31,697 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1960978685] 11:22:31,697 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,697 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 84.32 μs 11:22:31,697 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1960978685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:31,698 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 88.74 μs 11:22:31,699 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 changed state from IDLE to TX_OPEN 11:22:31,699 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:22:31,699 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 11:22:31,699 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:22:31,699 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 0 11:22:31,699 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 1 11:22:31,700 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@15501ad5 11:22:31,700 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1}, enqueuedTicks=805034910658} 11:22:31,700 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1}} 11:22:31,700 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 454 bytes, total slices to send: 1 11:22:31,700 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,700 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@7495ec17 11:22:31,700 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=805035356958} 11:22:31,702 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,702 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 11:22:31,702 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 11:22:31,702 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,702 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:31,702 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 11:22:31,702 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 11:22:31,702 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:31,702 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,702 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}]]] 11:22:31,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}]]] 11:22:31,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 11:22:31,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:22:31,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 11:22:31,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 11:22:31,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 11:22:31,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,707 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,707 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,707 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2dc621b5 11:22:31,708 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 11:22:31,708 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 11:22:31,708 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-570: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 11:22:31,710 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-570: Message size: 371 bytes, total slices to send: 1 11:22:31,710 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-570: Message does not need to be sliced - sending original message 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@2321392e 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 11:22:31,711 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:31,711 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 11:22:31,711 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 11:22:31,711 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1}, enqueuedTicks=805034910658, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 11:22:31,711 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0} 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=805035356958, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1} 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805029663737, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 11:22:31,717 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 1 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 11:22:31,717 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1f1d651a purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 11:22:31,717 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@6b7e1ca9 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 11:22:31,717 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1}, enqueuedTicks=805052373249} 11:22:31,717 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1}} 11:22:31,717 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 11:22:31,717 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 442 bytes, total slices to send: 1 11:22:31,717 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,717 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@b115282 11:22:31,717 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=805052702777} 11:22:31,719 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,719 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,719 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@112accb2 11:22:31,719 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 11:22:31,719 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 358 bytes, total slices to send: 1 11:22:31,719 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 11:22:31,721 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 11:22:31,721 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1}, enqueuedTicks=805052373249, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 11:22:31,721 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0} 11:22:31,721 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 11:22:31,721 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=805052702777, sessionId=1, txSequence=7} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 11:22:31,721 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1} 11:22:31,721 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 2 11:22:31,721 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@1c8e59d 11:22:31,721 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,721 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY}, enqueuedTicks=805056429393} 11:22:31,721 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY}} 11:22:31,721 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 291 bytes, total slices to send: 1 11:22:31,721 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,721 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 2 11:22:31,721 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@358c378d 11:22:31,721 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY}, enqueuedTicks=805056602327} 11:22:31,721 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY}} 11:22:31,721 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 292 bytes, total slices to send: 1 11:22:31,721 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,721 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 changed state from TX_OPEN to IDLE 11:22:31,721 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:22:31,721 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 changed state from IDLE to TX_OPEN 11:22:31,721 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:22:31,721 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} allocated sequence 0 11:22:31,721 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@3ec2b20c 11:22:31,721 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1, protocol=READY}, enqueuedTicks=805056841272} 11:22:31,721 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1, protocol=READY}} 11:22:31,722 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 428 bytes, total slices to send: 1 11:22:31,722 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,722 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:22:31,722 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 changed state from TX_OPEN to IDLE 11:22:31,722 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:22:31,722 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 3 11:22:31,722 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@1191196 11:22:31,722 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,722 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 ready 11:22:31,722 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 ready 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 11:22:31,723 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 to ready 11:22:31,723 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 to ready 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 ready 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 11:22:31,723 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 to ready 11:22:31,724 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 skipping transactions [2] 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY}, enqueuedTicks=805056602327, sessionId=1, txSequence=8} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY}, enqueuedTicks=805056429393, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1, protocol=READY}, enqueuedTicks=805056841272, sessionId=1, txSequence=9} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 11:22:31,725 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0} 11:22:31,725 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,725 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805060754792} 11:22:31,725 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=805060839574} 11:22:31,726 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}} 11:22:31,726 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 291 bytes, total slices to send: 1 11:22:31,726 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,726 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 3 11:22:31,726 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@4afcc209 11:22:31,726 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=805060967777} 11:22:31,726 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}} 11:22:31,726 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 292 bytes, total slices to send: 1 11:22:31,726 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 is already in state READY (READY) 11:22:31,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating canCommit 11:22:31,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 validated 11:22:31,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:31,728 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed canCommit 11:22:31,729 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipping transactions [[2..2]] 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.SkipTransactionsPayload, isSendImmediate: false 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 is already in state READY (READY) 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating canCommit 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 validated 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed canCommit 11:22:31,730 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 11:22:31,737 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 11:22:31,737 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=805060967777, sessionId=1, txSequence=10} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 11:22:31,737 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} canCommit completed 11:22:31,737 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 11:22:31,737 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=805060839574, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 11:22:31,737 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} canCommit completed 11:22:31,737 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 4 11:22:31,737 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@b289d83 11:22:31,737 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805072569343} 11:22:31,737 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 4 11:22:31,737 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@60993d8d 11:22:31,737 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805072674806} 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating preCommit 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 candidate ready 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - messages: [] 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 prepared 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed preCommit 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating preCommit 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 candidate ready 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - candidate rootPath: / 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - messages: [] 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: preCommit - successfulFromPrevious: [] 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 prepared 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:31,738 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed preCommit 11:22:31,739 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 11:22:31,739 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805072569343, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 11:22:31,739 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} preCommit completed, clearing successfulRequests 11:22:31,739 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 11:22:31,739 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805072674806, sessionId=1, txSequence=11} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 11:22:31,739 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} preCommit completed, clearing successfulRequests 11:22:31,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 5 11:22:31,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@552fea9a 11:22:31,748 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805083119526} 11:22:31,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 5 11:22:31,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@1a24a139 11:22:31,748 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805083224668} 11:22:31,748 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating commit 11:22:31,748 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 submitted to persistence 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating commit 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 225 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 11:22:31,749 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 submitted to persistence 11:22:31,750 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 11:22:31,750 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 11:22:31,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 11:22:31,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 11:22:31,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:22:31,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 11:22:31,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=236} 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=225} 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 11:22:31,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2240/0x00000008016b5bf8@77ae918d 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@204c52fa 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 11:22:31,760 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 11:22:31,760 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: commit - successfulFromPrevious: [] 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 11:22:31,761 AM [cluster-test-shard-dispatcher-26] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 11:22:31,761 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 11:22:31,761 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 11:22:31,761 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:22:31,761 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,761 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 11:22:31,762 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 84.14 μs 11:22:31,762 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,762 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 23.56 μs 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805060754792, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 confirmed transaction skip 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805030485836, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE}: purge completed 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: purge completed 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@10d75564 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805083119526, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} doCommit completed 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 6 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@493acca8 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805105651354} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 11:22:31,770 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805083224668, sessionId=1, txSequence=12} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 11:22:31,771 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} doCommit completed 11:22:31,771 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} 11:22:31,771 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED}: initiating purge 11:22:31,771 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} allocated sequence 6 11:22:31,771 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@1b28a6ed 11:22:31,771 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:31,771 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805105925340} 11:22:31,771 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} allocated sequence 1 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 11:22:31,771 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@4ba03100 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 11:22:31,771 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=SIMPLE}, enqueuedTicks=805106150555} 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 11:22:31,771 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=SIMPLE}} 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 11:22:31,771 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 292 bytes, total slices to send: 1 11:22:31,771 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 11:22:31,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 11:22:31,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 11:22:31,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 11:22:31,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 11:22:31,772 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,772 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 51.25 μs 11:22:31,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 11:22:31,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 11:22:31,772 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,772 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 42.99 μs 11:22:31,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 11:22:31,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 11:22:31,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 is already in state READY (READY) 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct canCommit 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 validated 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:31,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct preCommit 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 candidate ready 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - candidate rootPath: / 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - messages: [] 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: preCommit - successfulFromPrevious: [] 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 prepared 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:31,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct commit 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 2 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 4 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 11:22:31,776 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 submitted to persistence 11:22:31,777 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 11:22:31,777 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 11:22:31,777 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=222} 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 11:22:31,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 5, nextIndex: 6 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 3 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@f0f83f8 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: commit - successfulFromPrevious: [] 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 11:22:31,778 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 11:22:31,778 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 54.24 μs 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:22:31,778 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 11:22:31,779 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 11:22:31,779 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805105925340, sessionId=1, txSequence=13} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 11:22:31,779 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE}: purge completed 11:22:31,779 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} 11:22:31,779 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 11:22:31,779 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=SIMPLE}, enqueuedTicks=805106150555, sessionId=1, txSequence=14} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 11:22:31,780 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} directCommit completed 11:22:31,780 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED}: initiating purge 11:22:31,780 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} allocated sequence 2 11:22:31,780 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@43bd5fea 11:22:31,780 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 11:22:31,780 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 11:22:31,780 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 11:22:31,780 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:31,780 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 invoking destroy 11:22:31,780 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805114892106} 11:22:31,780 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805115023889} 11:22:31,780 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c invoking destroy 11:22:31,780 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805115127581} 11:22:31,780 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2 changed state from IDLE to CLOSED 11:22:31,780 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4bf96089: creating a new snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 11:22:31,780 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 11:22:31,780 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 11:22:31,780 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:31,780 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@a3394f3 11:22:31,780 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=805115539021} 11:22:31,782 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,782 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 11:22:31,782 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 11:22:31,782 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:31,782 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,782 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,784 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,784 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,784 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,784 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,784 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,784 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,784 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 11:22:31,784 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 11:22:31,784 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 11:22:31,784 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 11:22:31,784 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-570: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 11:22:31,785 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-570: Message size: 370 bytes, total slices to send: 1 11:22:31,785 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-570: Message does not need to be sliced - sending original message 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 5 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 3, lastIndex(): 5 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 11:22:31,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 11:22:31,786 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 11:22:31,786 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=805115539021, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 11:22:31,786 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0} 11:22:31,786 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 11:22:31,786 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 11:22:31,786 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:22:31,786 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@710374d1 11:22:31,786 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=805121226412} 11:22:31,786 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 11:22:31,786 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 127.6 μs 11:22:31,786 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,786 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 6 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 11:22:31,786 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 11:22:31,787 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,787 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 11:22:31,787 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 11:22:31,787 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 232 bytes, total slices to send: 1 11:22:31,787 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 11:22:31,787 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 11:22:31,788 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 7, nextIndex: 8 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 5 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@237fe4a2 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 7 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2241/0x00000008016b6668@2f11ac6b 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:22:31,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=805121226412, sessionId=1, txSequence=17} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805114892106, sessionId=1, txSequence=15} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 11:22:31,793 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE}: purge completed 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805115127581, sessionId=1, txSequence=16} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1} 11:22:31,793 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4ce3826c 11:22:31,794 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:31,794 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac transmitting entry ConnectionEntry{request=PurgeLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805129327808} 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 7 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 5, lastIndex(): 7 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 7 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 11:22:31,794 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 11:22:31,796 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:31,796 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 11:22:31,796 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 11:22:31,796 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 11:22:31,796 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 11:22:31,796 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ce2d43b 11:22:31,796 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 11:22:31,796 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 11:22:31,796 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3ccb2dc7 11:22:31,797 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 11:22:31,797 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,797 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3ccb2dc7 11:22:31,797 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ce2d43b 11:22:31,797 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 11:22:31,797 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 11:22:31,797 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 11:22:31,797 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:22:31,797 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:31,798 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 11:22:31,798 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 7, lastIndex: 7, isVoting: true 11:22:31,798 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:22:31,798 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 11:22:31,798 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 11:22:31,798 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=12, message=PurgeLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1901361926]}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 11:22:31,798 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: request Envelope{sessionId=1, txSequence=12, message=PurgeLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1901361926]}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:31,799 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:31,799 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:22:31,799 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:31,799 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:31,799 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:31,799 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:31,799 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:31,799 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:31,800 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@56a240c8, splicing queue 11:22:31,800 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:22:31,800 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad4f5ac queue spliced 1 messages 11:22:31,800 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:31,800 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 11:22:31,800 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 11:22:31,800 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 11:22:31,800 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:22:31,800 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 2 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2094654634] 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5d11fc1f, splicing queue 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c4c0e4c queue spliced 2 messages 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 1 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:31,803 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2094654634] 11:22:31,804 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:22:31,804 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:22:31,804 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:31,804 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 6, nextIndex: 7 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@6d361b5a 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2241/0x00000008016b6668@1d7f92d9 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 6, lastIndex: 6, isVoting: true 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 11:22:31,805 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 6 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 11:22:31,808 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 11:22:31,809 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 11:22:31,809 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 11:22:31,809 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} not found in inflight queue, checking pending queue 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805105651354, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926]}, enqueuedTicks=805115023889, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1} 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5e597bf2: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a714144 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:31,809 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:32,15 AM [cluster-test-shard-dispatcher-30] [WARN] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 203, lastApplied : 6, commitIndex : 6 11:22:32,213 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:22:32,213 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:32,214 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 11:22:32,214 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-574: Closing 11:22:32,214 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:22:32,214 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 11:22:32,214 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-575: Closing 11:22:32,214 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:22:32,214 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 11:22:32,215 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:22:32,215 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 11:22:32,215 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 11:22:32,215 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:22:32,215 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:22:32,215 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-897102262] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-897102262] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:32,216 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:32,216 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:32,221 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:32,221 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:32,221 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:32,222 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:32,224 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:32,234 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 11:22:32,234 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 11:22:32,234 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 11:22:32,234 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 11:22:32,234 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 11:22:32,234 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:22:32,235 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 11:22:32,235 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 11:22:32,235 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:32,235 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:33,228 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8219749090283494651] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:33,247 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:33,247 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:33,248 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1869553224039645473] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:33,249 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:33.250] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:33,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:33,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:22:33.261] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:33,274 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:33,281 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3712881177307761505] 11:22:33,281 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:33,282 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:33,282 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:33,282 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:33,282 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:33,292 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:33,292 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:33,292 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:33,303 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:33,330 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4502501523585375765] 11:22:33,330 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:33,333 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:33,333 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:33,336 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:33,336 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:33,362 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:33,377 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [765154330815460940] 11:22:33,377 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:33,380 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:33,380 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:33,380 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:33,380 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:33,390 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:33,391 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:33,393 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 11:22:33,398 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 11:22:33,398 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,400 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:33,403 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:33,412 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 11:22:33,412 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 11:22:33,412 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 11:22:33,412 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 11:22:33,412 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7df2de3e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11f21ba2 11:22:33,412 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@54f087f8 11:22:33,412 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:33,412 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:33,412 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1709032953412, None)) 11:22:33,412 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032953411,0,0)) 11:22:33,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@62cb4ff7 11:22:33,413 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11f21ba2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@146795e6 11:22:33,413 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 11:22:33,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:33,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 11:22:33,413 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:33,415 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:33,418 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 11:22:33,420 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:33,420 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:33,420 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226] 11:22:33,421 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 11:22:33,421 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 11:22:33,421 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,421 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-1929446672 created and ready for shard:member-1-shard-cars-testInstallSnapshot 11:22:33,421 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-1929446672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,421 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#1070624375] 11:22:33,422 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#1070624375] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#870691563] 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1283899093] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 11:22:33,422 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1283899093] 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 458.9 μs 11:22:33,422 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 610.7 μs: snapshotIndex=5, snapshotTerm=1, journal-size=0 11:22:33,423 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 11:22:33,423 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1283899093] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,423 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:33,423 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 204.4 μs 11:22:33,423 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 1.087 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 11:22:33,423 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 11:22:33,423 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 11:22:33,423 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 11:22:33,423 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 11:22:33,423 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 11:22:33,423 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 11:22:33,423 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 11:22:33,430 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1309248718] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,430 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 11:22:33,435 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 11:22:33,435 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 11:22:33,435 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 11:22:33,435 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@ded33b1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5f12b37a 11:22:33,435 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 11:22:33,435 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:33,435 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:33,438 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@72e99bfd 11:22:33,438 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1709032953435, None)) 11:22:33,438 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032953434,0,0)) 11:22:33,438 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5f12b37a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6c41d13 11:22:33,438 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 11:22:33,438 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2c2dfd77 11:22:33,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:33,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 11:22:33,439 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:33,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:22:33,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-2-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#929760738] 11:22:33,440 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 11:22:33,440 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#929760738] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,440 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-11058541 created and ready for shard:member-2-shard-cars-testInstallSnapshot 11:22:33,440 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-11058541] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,440 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-151491815] 11:22:33,440 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 11:22:33,440 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-151491815] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:33,441 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:33,441 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:33,441 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 11:22:33,441 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 11:22:33,441 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 11:22:33,441 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 11:22:33,441 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 11:22:33,441 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:33,441 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 11:22:33,441 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 11:22:33,454 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:33,456 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:33,489 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] 11:22:33,489 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 11:22:33,489 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226] 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 11:22:33,652 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 11:22:33,652 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 11:22:33,653 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 11:22:33,972 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 11:22:33,972 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 11:22:33,972 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 11:22:33,974 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:34,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 11:22:34,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 11:22:34,292 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 11:22:34,293 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:34,293 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:34,293 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:34,293 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 11:22:34,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:34,294 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:34,294 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:22:34,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:34,294 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:22:34,305 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:34,305 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 11:22:34,305 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:22:34,305 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:34,305 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:22:34,305 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 11:22:34,396 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:34,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 11:22:34,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 11:22:34,572 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 11:22:34,572 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 11:22:34,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:22:34,575 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:22:34,575 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:22:34,575 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 11:22:34,575 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:22:34,575 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:22:34,575 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 11:22:34,575 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 11:22:34,575 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 11:22:34,576 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 11:22:34,576 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:34,576 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:34,576 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 11:22:34,576 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 11:22:34,576 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 11:22:34,576 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 11:22:34,576 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 11:22:34,576 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:34,576 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:34,576 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 11:22:34,576 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 11:22:34,577 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 11:22:34,577 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:22:34,577 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 11:22:34,577 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 11:22:34,577 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 11:22:34,577 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 11:22:34,577 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 11:22:34,579 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 11:22:34,579 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@45ad5858, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 11:22:34,579 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 11:22:34,579 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 11:22:34,579 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 11:22:34,580 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 11:22:34,580 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 11:22:34,580 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 11:22:34,580 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 11:22:34,580 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 11:22:34,580 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 11:22:34,580 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 11:22:34,580 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 11:22:34,580 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 11:22:34,580 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 11:22:34,581 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@70a0e6fa, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 11:22:34,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 11:22:34,581 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 11:22:34,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 11:22:34,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 11:22:34,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 11:22:34,581 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 11:22:34,582 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 11:22:34,582 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1229498488] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 11:22:34,582 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1229498488] 11:22:34,582 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 367.0 μs 11:22:34,582 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 11:22:34,582 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 11:22:34,582 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 11:22:34,582 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1229498488] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:34,582 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 11:22:34,582 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 158.6 μs 11:22:34,583 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 11:22:34,583 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 11:22:34,583 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 11:22:34,583 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 11:22:34,595 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@462fb919: creating a new snapshot member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 11:22:34,596 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 11:22:34,596 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:34,596 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:34,596 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] 11:22:34,596 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=0} shard 1 11:22:34,596 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 11:22:34,596 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:34,596 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#135019586], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@74d4ea44 11:22:34,596 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:22:34,596 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:34,596 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5303285f] 11:22:34,597 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5303285f] 11:22:34,597 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@615cf0b4, attempting to connect 11:22:34,597 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} is not yet known 11:22:34,597 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, nanosAgo=10611, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 11:22:34,597 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:22:34,597 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:34,597 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5bd66b78 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3d6fffb4 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3d6fffb4 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#135019586], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@462fb919 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@30ab4757 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7bf174ff superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3921a66a, splicing queue 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7bf174ff queue spliced 0 messages 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 678.5 μs 11:22:34,598 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0} 11:22:34,599 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 11:22:34,599 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 11:22:34,599 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 11:22:34,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 11:22:34,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 11:22:34,599 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 11:22:34,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1aa343e9 11:22:34,599 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:34,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1aa343e9 11:22:34,599 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:22:34,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 11:22:34,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:22:34,599 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@782079a1 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@3ff3d0cf, splicing queue 11:22:34,600 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@782079a1 queue spliced 0 messages 11:22:34,600 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:34,600 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: refreshing backend for shard 1 11:22:34,600 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:34,600 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:34,600 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:34,600 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] 11:22:34,600 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:34,600 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testInstallSnapshot, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 11:22:34,600 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 11:22:34,600 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 11:22:34,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:34,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:34,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testInstallSnapshot, matchIndex: 5, lastIndex: 5, isVoting: true 11:22:34,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Follower's log matches - sending ElectionTimeout 11:22:34,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 11:22:34,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leader transfer complete 11:22:34,814 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-1-shard-cars-testInstallSnapshot (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-2-shard-cars-testInstallSnapshot, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 211, lastApplied : 5, commitIndex : 5 11:22:35,12 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 11:22:35,12 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] ShardManager - shard-manager-testInstallSnapshot: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:35,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Received ShardNotInitializedTimeout message for shard cars 11:22:35,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Returning NoShardLeaderException for shard cars 11:22:35,12 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:35,13 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:35,14 AM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:35,14 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:35,14 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 11:22:35,14 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-580: Closing 11:22:35,14 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 11:22:35,14 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 11:22:35,15 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 11:22:35,15 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 11:22:35,15 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 11:22:35,15 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: shutting down 11:22:35,15 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 11:22:35,16 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:35,16 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:35,16 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:35,16 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-677363492] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:35,17 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:35,17 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:35,22 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:35,22 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:35,22 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:35,22 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:35,23 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:35,33 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 11:22:35,33 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 11:22:35,33 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 11:22:35,33 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 11:22:35,34 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 11:22:35,34 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:35,34 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:36,36 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3712881177307761505] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:36,56 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4502501523585375765] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:36,56 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:36,56 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:36,58 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:36.059] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:36,64 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:36,64 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:36,67 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:36.067] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:36,80 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:36,86 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4850369663335322956] 11:22:36,86 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:36,87 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:36,88 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:36,93 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:36,93 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:36,97 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:36,97 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:36,97 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:36,102 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:36,112 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-774977545471663913] 11:22:36,113 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:36,116 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:36,116 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:36,116 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:36,116 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:36,135 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:22:36,141 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5266987714269423228] 11:22:36,141 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:36,145 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:36,145 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:36,145 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:36,145 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:36,153 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:36,154 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:36,156 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 11:22:36,162 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:36,162 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,164 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 11:22:36,164 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:22:36,164 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5aa895dc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@39236c74 11:22:36,164 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3d1cef88 11:22:36,165 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709032956164, None)) 11:22:36,165 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032956163,0,0)) 11:22:36,165 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@39236c74 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@55575858 11:22:36,165 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:36,165 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,165 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:36,165 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,165 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@579b3ace 11:22:36,168 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 11:22:36,168 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:36,168 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,168 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:36,170 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:36,172 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 11:22:36,175 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,175 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,175 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] 11:22:36,175 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 11:22:36,175 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,176 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#-1592908384] 11:22:36,177 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-136666368] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,177 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#2006614899] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,179 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:36,181 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:36,182 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#2006614899 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,182 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 11:22:36,184 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#-1592908384] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,184 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:36,184 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:36,184 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:36,184 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 11:22:36,185 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 11:22:36,185 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 11:22:36,186 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 11:22:36,186 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 11:22:36,187 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:36,188 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 11:22:36,189 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:36,191 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:36,192 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 11:22:36,192 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:22:36,192 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7b154124 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a84d96f 11:22:36,192 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5364b708 11:22:36,192 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:36,192 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,192 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709032956192, None)) 11:22:36,192 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:36,192 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,192 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032956191,0,0)) 11:22:36,193 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@394b5df9 11:22:36,193 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a84d96f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2180b712 11:22:36,193 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 11:22:36,193 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:36,193 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,193 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,193 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,193 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1388967561] 11:22:36,194 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 11:22:36,194 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 11:22:36,194 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1388967561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,194 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-155573039] 11:22:36,194 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#911396840] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,194 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#911396840 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,195 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-155573039] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:36,199 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:36,199 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:36,199 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 11:22:36,199 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 11:22:36,199 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:36,199 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 11:22:36,199 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,199 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 11:22:36,199 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 11:22:36,199 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 11:22:36,243 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] 11:22:36,243 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 11:22:36,244 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:36,429 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 11:22:36,429 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 11:22:36,430 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 11:22:36,699 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 11:22:36,699 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:36,699 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 11:22:36,778 AM [cluster-test-shard-dispatcher-19] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:37,9 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 11:22:37,9 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,9 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 11:22:37,109 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:37,109 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,110 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,120 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:37,120 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,121 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:37,121 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,121 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,121 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,161 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:37,278 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 11:22:37,278 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,279 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 11:22:37,279 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,279 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,280 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:37,280 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:37,280 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 11:22:37,281 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:37,281 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:37,281 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 11:22:37,281 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:37,281 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,281 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:22:37,282 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 11:22:37,282 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 11:22:37,283 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 11:22:37,283 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:37,283 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:37,283 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 11:22:37,283 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 11:22:37,283 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:37,283 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:37,283 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 11:22:37,283 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:37,283 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:37,283 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 11:22:37,283 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 11:22:37,283 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:37,284 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 11:22:37,284 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:37,301 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:37,303 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:37,306 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 11:22:37,309 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-348716997] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:37,310 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:37,311 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 11:22:37,311 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:22:37,311 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@39075964 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@78b3b8f3 11:22:37,311 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5d57e76 11:22:37,311 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709032957311, None)) 11:22:37,312 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032957310,0,0)) 11:22:37,312 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@78b3b8f3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@28b65a7f 11:22:37,312 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 11:22:37,313 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:37,313 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:37,314 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:37,314 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:37,324 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:37,324 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:37,324 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:37,324 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:37,326 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:37,346 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] 11:22:37,346 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 11:22:37,346 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] 11:22:37,346 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] 11:22:37,400 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 11:22:38,131 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:22:38,329 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-5266987714269423228] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:38,340 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-5266987714269423228] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:43,230 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 11:22:43,230 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:43,230 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:43,251 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:43,251 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 11:22:43,371 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-136666368] 11:22:43,371 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 11:22:43,371 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1388967561] 11:22:43,371 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] 11:22:43,371 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:43,371 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:22:43,372 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:22:43,372 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1388967561] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:22:43,372 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1388967561] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 11:22:43,372 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,372 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 11:22:43,372 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,373 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979] 11:22:43,373 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:22:43,373 AM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:43,374 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:43,374 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:22:43,374 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:43,374 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 11:22:43,374 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 11:22:43,374 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6df0a0a4 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6df0a0a4 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:43,375 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:43,381 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 11:22:43,381 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 11:22:43,381 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:43,381 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:43,382 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 11:22:43,382 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 11:22:43,382 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:43,382 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:43,382 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 11:22:43,789 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 11:22:43,789 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-239157979]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:43,789 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:43,789 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-585: Closing 11:22:43,789 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 11:22:43,790 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:43,790 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:43,790 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 11:22:43,790 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 11:22:43,790 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:43,790 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,790 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,790 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-266355992] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-266355992] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,790 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:43,791 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#73295990] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:43,791 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:43,791 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:44,338 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:44,339 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 11:22:44,749 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [akka.cluster.ddata.Replicator$Internal$Status] from Actor[akka://cluster-test/system/clusterReceptionist/replicator#-407786600] to Actor[akka://cluster-test/deadLetters] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,749 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [akka.cluster.ddata.Replicator$Internal$Status] from Actor[akka://cluster-test/system/clusterReceptionist/replicator#-407786600] to Actor[akka://cluster-test/deadLetters] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,809 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4850369663335322956] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:44,812 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:44,812 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:22:44.814] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:44,820 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 11:22:44,820 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 11:22:44,820 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 11:22:44,820 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 11:22:44,821 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 11:22:44,821 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:44,821 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:44,825 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:44.826] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:44,848 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:44,853 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7882673773915781463] 11:22:44,853 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:44,860 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:44,860 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:44,860 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:44,860 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:44,866 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:44,866 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:44,866 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:44,869 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:44,873 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5411668210476788372] 11:22:44,873 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:44,875 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:44,875 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:44,875 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:44,875 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:44,891 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:44,899 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7508908619998722717] 11:22:44,899 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:44,909 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:44,909 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:44,917 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:44,918 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:44,919 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 11:22:44,926 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:44,926 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:44,928 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1168359264] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,928 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:22:44,928 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 11:22:44,928 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:22:44,928 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@728d5712 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a474973 11:22:44,928 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3e78dae3 11:22:44,929 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:44,929 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709032964928, None)) 11:22:44,930 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032964927,0,0)) 11:22:44,930 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a474973 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5a129fff 11:22:44,930 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 11:22:44,930 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:44,931 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:44,933 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 11:22:44,937 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 11:22:44,937 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,937 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:44,937 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,937 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2597decc 11:22:44,937 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:44,937 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,938 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,938 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,938 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109] 11:22:44,939 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 11:22:44,939 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#247208666 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,940 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-1984860588] 11:22:44,940 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,941 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#247208666] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,941 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-1984860588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,949 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:44,951 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:44,953 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:22:44,953 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:44,954 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 11:22:44,954 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1795857382] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,954 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:44,954 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:44,954 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:44,954 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 11:22:44,954 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 11:22:44,954 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 11:22:44,954 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 11:22:44,954 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 11:22:44,955 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 11:22:44,958 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 11:22:44,958 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 11:22:44,958 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:22:44,958 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,958 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6ce57ad5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11000f7 11:22:44,958 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@44128eb1 11:22:44,958 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709032964958, None)) 11:22:44,958 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:44,958 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,959 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032964957,0,0)) 11:22:44,959 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@394e905e 11:22:44,959 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11000f7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@748a1ed8 11:22:44,959 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 11:22:44,959 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:44,959 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,960 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,960 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,960 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-194299978] 11:22:44,960 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 11:22:44,960 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-194299978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,960 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1678375932 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,960 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1678375932] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,960 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#-1799566330] 11:22:44,960 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#-1799566330] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:44,960 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 11:22:44,961 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:44,961 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:44,961 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 11:22:44,961 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 11:22:44,961 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:44,961 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 11:22:44,961 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:44,961 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 11:22:44,961 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 11:22:44,961 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 11:22:45,9 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1168359264] 11:22:45,9 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 11:22:45,10 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109] 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:45,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 11:22:45,257 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 11:22:45,257 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 11:22:45,497 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:45,545 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:22:45,546 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,546 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 11:22:45,846 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:22:45,846 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,846 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 11:22:45,877 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:45,877 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,877 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,877 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:45,877 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,877 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,877 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,877 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:45,877 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:45,887 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:45,887 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,887 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,887 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:45,887 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,887 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:45,927 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:46,146 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:22:46,146 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,146 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 11:22:46,146 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:46,149 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:46,149 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:46,150 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 11:22:46,150 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:46,150 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:46,150 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:22:46,150 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 11:22:46,151 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 11:22:46,151 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,151 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 11:22:46,151 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 11:22:46,151 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:46,151 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:46,151 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 11:22:46,151 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 11:22:46,152 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 11:22:46,152 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:46,152 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 11:22:46,152 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:46,152 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:46,152 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 11:22:46,152 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 11:22:46,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1795857382] 11:22:46,216 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 11:22:46,216 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-194299978] 11:22:46,216 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1795857382] 11:22:46,216 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 11:22:46,216 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-194299978] 11:22:46,228 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,228 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,228 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 11:22:46,231 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 11:22:46,231 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:46,231 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:46,231 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:22:46,231 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:46,231 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-245-0 of 1 operations, client version 13 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-245-0 ready 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 validated 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-245-0: success 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 candidate ready 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: canCommit - candidate rootPath: / 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: canCommit - messages: [] 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: preCommit - successfulFromPrevious: [] 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 prepared 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:46,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-245-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,233 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 submitted to persistence 11:22:46,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:46,234 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:46,237 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:22:46,237 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:22:46,237 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:46,237 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:22:46,237 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:46,237 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 11:22:46,238 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=7687187} is already disabled 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:46,239 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 11:22:46,239 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 11:22:46,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 11:22:46,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: commit - successfulFromPrevious: [] 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from COMMIT_PENDING to COMMITTED 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-764#-1981965791] 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-2023416197] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:46,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:22:46,241 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-2023416197] 11:22:46,241 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:46,241 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-2023416197] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:46,242 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 134.5 μs 11:22:46,242 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 11:22:46,242 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:46,242 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 11:22:46,242 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:22:46,242 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 11:22:46,242 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d3ff96a: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 11:22:46,243 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-245-0 11:22:46,243 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 11:22:46,243 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:46,243 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:46,243 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1168359264] 11:22:46,243 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 11:22:46,243 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 11:22:46,243 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:46,243 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475647085], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@9a2dda8 11:22:46,243 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1350667283] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 11:22:46,243 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:22:46,243 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:22:46,243 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:22:46,243 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:46,243 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@74662522] 11:22:46,243 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1350667283] 11:22:46,243 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@74662522] 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:46,244 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7968cefe, attempting to connect 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1350667283] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:46,244 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} is not yet known 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 135.4 μs 11:22:46,244 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, nanosAgo=6570, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:22:46,244 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@233621ab instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7b7a4348 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7b7a4348 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475647085], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d3ff96a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4acbbd77 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@321cd90f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1b55bc07, splicing queue 11:22:46,244 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@321cd90f queue spliced 0 messages 11:22:46,245 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 458.7 μs 11:22:46,245 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 11:22:46,245 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,245 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,245 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 11:22:46,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-246-0 of 1 operations, client version 13 11:22:46,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-246-0 ready 11:22:46,251 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 canCommit 11:22:46,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109)] resolved 11:22:46,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: successfully resolved all cohort actors 11:22:46,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 finishCanCommit 11:22:46,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-246-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109)] 11:22:46,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109)] 11:22:46,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 validated 11:22:46,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:46,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-246-0: success 11:22:46,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:22:46,251 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit returning result: true 11:22:46,252 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 commit 11:22:46,252 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 finish commit 11:22:46,252 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-246-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109)] 11:22:46,252 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109)] 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 candidate ready 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit - candidate rootPath: / 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit - messages: [] 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: preCommit - successfulFromPrevious: [] 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 prepared 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-246-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 11:22:46,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 submitted to persistence 11:22:46,253 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:46,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 11:22:46,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:22:46,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:22:46,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 11:22:46,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:46,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 11:22:46,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: commit - successfulFromPrevious: [] 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from COMMIT_PENDING to COMMITTED 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-1025612109$a] 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:46,254 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:22:46,254 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: commit succeeded 11:22:46,255 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:46,255 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 11:22:46,255 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 164.3 μs 11:22:46,255 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d3ff96a: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 11:22:46,255 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 11:22:46,255 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:22:46,255 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,255 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 11:22:46,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 11:22:46,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 11:22:46,256 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:22:46,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2612f8b5 11:22:46,256 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:46,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2612f8b5 11:22:46,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 11:22:46,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27cd334b superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@23f2a36a, splicing queue 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27cd334b queue spliced 0 messages 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: refreshing backend for shard 1 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:46,256 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1168359264] 11:22:46,257 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:46,257 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 11:22:46,257 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-246-0 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:22:46,258 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 11:22:46,258 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 11:22:46,259 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:22:46,259 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:46,259 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:46,259 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 11:22:46,259 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 11:22:46,259 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:46,259 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 258.1 μs 11:22:46,259 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 11:22:46,666 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 11:22:46,666 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:46,667 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,667 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-591: Closing 11:22:46,667 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:22:46,667 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:22:46,667 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:22:46,667 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 11:22:46,668 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 11:22:46,668 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:22:46,668 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:22:46,669 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:46,669 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:46,674 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:46,674 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:46,674 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:46,674 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:46,675 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:46,690 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 11:22:46,691 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 11:22:46,691 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 11:22:46,691 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:22:46,691 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 11:22:46,692 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:46,692 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:47,685 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7882673773915781463] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:47,706 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5411668210476788372] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:47,708 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:47,708 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:47,709 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:47.711] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:47,718 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:47,718 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:47,721 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:47.721] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:47,732 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:22:47,738 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-399232174813925204] 11:22:47,738 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:47,739 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:47,739 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:47,740 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:47,740 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:47,749 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:47,749 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:47,749 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:47,753 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:47,759 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4625987186746434566] 11:22:47,759 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:47,762 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:47,762 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:47,768 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:47,768 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:47,775 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:47,788 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [79384885486725109] 11:22:47,789 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:47,791 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:47,791 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:47,797 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:47,808 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 11:22:47,808 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#222396366)] 11:22:47,808 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 11:22:47,808 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1017014274)] 11:22:47,809 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6954b08a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 1 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@693e16ee 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1384729715], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:47,809 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2d4163fc aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 1 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1901361926], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@67f9e5b9 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-430899547], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:22:47,809 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0} 11:22:47,809 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5bd66b78 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#135019586], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@63c67d9e 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 11:22:47,809 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3d6fffb4 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#135019586]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@26c4ef82 11:22:47,809 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2105393226], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:22:47,810 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:47,810 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:47,810 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@233621ab aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475647085], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@1728eabc 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 11:22:47,810 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7b7a4348 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475647085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@47d24f4a 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:22:47,810 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7b7a4348 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475647085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@46eb10cd 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 11:22:47,810 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1025612109], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:22:47,811 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:47,814 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:47,815 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:47,816 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 11:22:47,829 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:22:47,830 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-22950521] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,832 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:22:47,832 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:22:47,832 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@575e4d26 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2efc780e 11:22:47,832 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7bacc01c 11:22:47,832 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 11:22:47,832 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,833 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:47,833 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,833 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709032967833, None)) 11:22:47,833 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032967832,0,0)) 11:22:47,833 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@41877409 11:22:47,833 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2efc780e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@20674d30 11:22:47,834 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 11:22:47,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:47,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,834 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:47,834 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506] 11:22:47,835 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 11:22:47,835 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 11:22:47,836 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:47,837 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 11:22:47,839 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-28663475 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,839 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#277233624] 11:22:47,839 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,839 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-28663475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,839 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#277233624] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,840 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:47,840 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:47,840 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 11:22:47,840 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 11:22:47,840 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:47,840 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 11:22:47,840 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 11:22:47,840 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 11:22:47,840 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 11:22:47,844 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:22:47,846 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1192963139] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,857 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:22:47,857 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:22:47,857 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1351697d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57cb3c87 11:22:47,857 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:47,857 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@12c9b81a 11:22:47,857 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 11:22:47,857 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,858 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:47,858 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,858 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709032967858, None)) 11:22:47,858 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032967857,0,0)) 11:22:47,858 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57cb3c87 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@296b371c 11:22:47,859 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 11:22:47,859 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:47,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@74a012c6 11:22:47,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:47,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,862 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,862 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,862 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-173732814] 11:22:47,862 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 11:22:47,863 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-173732814] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,863 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-913092126] 11:22:47,863 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-170028766 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,863 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-913092126] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,863 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 11:22:47,863 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-170028766] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:47,865 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:47,865 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:47,865 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:47,865 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 11:22:47,865 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 11:22:47,865 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:47,865 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 11:22:47,865 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 11:22:47,866 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 11:22:47,866 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 11:22:47,909 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-22950521] 11:22:47,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 11:22:47,909 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506] 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:48,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 11:22:48,91 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:22:48,92 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 11:22:48,321 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:48,321 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,321 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 11:22:48,410 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:48,641 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:48,641 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,641 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 11:22:48,751 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:48,751 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:48,752 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:48,752 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,752 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,752 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:48,752 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,752 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,752 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,764 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:48,764 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,764 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,764 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:48,764 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,764 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:48,871 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:48,871 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,871 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 11:22:48,871 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,872 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:48,872 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 11:22:48,872 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:48,872 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:48,872 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 11:22:48,873 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:48,873 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,873 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:48,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:48,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:48,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 11:22:48,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:48,876 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:48,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:22:48,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 11:22:48,876 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:22:48,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 11:22:48,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,877 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 11:22:48,877 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:22:48,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:48,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:48,877 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 11:22:48,878 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:22:48,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:48,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:48,878 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 11:22:48,878 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 11:22:48,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:22:48,878 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:48,920 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b2f38e5: creating a new transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:48,920 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 11:22:48,920 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:48,920 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:48,920 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1192963139] 11:22:48,920 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:48,921 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 11:22:48,921 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 11:22:48,921 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:48,921 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:48,921 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 11:22:48,921 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 11:22:48,921 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@6ef00540 11:22:48,921 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:22:48,921 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@47f33e44 11:22:48,922 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@30a7b13c, attempting to connect 11:22:48,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} is not yet known 11:22:48,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=11771, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:22:48,929 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@787aadb4 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2c12a363 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2c12a363 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:48,929 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2257/0x00000008016c5568@5e94627f 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3}, enqueuedTicks=822256497306} 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3}} 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: Message size: 467 bytes, total slices to send: 1 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: Message does not need to be sliced - sending original message 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@44b30ba0 11:22:48,930 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=822256497306} 11:22:48,931 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b2f38e5 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:48,931 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 11:22:48,931 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6bea065 11:22:48,931 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@43833f5a superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4dadeb7d, splicing queue 11:22:48,931 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@43833f5a queue spliced 0 messages 11:22:48,931 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.744 ms 11:22:48,932 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-593: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 11:22:48,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-593: Message size: 372 bytes, total slices to send: 1 11:22:48,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-593: Message does not need to be sliced - sending original message 11:22:48,934 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 11:22:48,934 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3}, enqueuedTicks=822256497306, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 11:22:48,934 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=3} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 11:22:48,934 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 11:22:48,934 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=822256497306, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 11:22:48,934 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1} 11:22:48,934 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:22:48,934 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:22:48,934 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@74ee438b 11:22:48,934 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 11:22:48,934 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 2 11:22:48,935 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303698@1a5a8253 11:22:48,935 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 11:22:48,935 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@44858e42 11:22:48,935 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:48,935 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=1}, enqueuedTicks=822270025777} 11:22:48,935 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=1}} 11:22:48,935 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: Message size: 457 bytes, total slices to send: 1 11:22:48,935 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: Message does not need to be sliced - sending original message 11:22:48,935 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 3 11:22:48,935 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1237/0x0000000801303000@24243b25 11:22:48,935 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=822270441276} 11:22:48,943 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=1} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 11:22:48,943 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=1}, enqueuedTicks=822270025777, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 11:22:48,943 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2} 11:22:48,943 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 11:22:48,943 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=822270441276, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 11:22:48,943 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true} 11:22:48,943 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b2f38e5 readied transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:48,943 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} allocated sequence 4 11:22:48,943 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@44529c8e 11:22:48,943 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:48,943 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=0, protocol=SIMPLE}, enqueuedTicks=822278400099} 11:22:48,943 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=0, protocol=SIMPLE}} 11:22:48,943 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: Message size: 289 bytes, total slices to send: 1 11:22:48,943 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-594: Message does not need to be sliced - sending original message 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transitioned member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 11:22:48,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 250 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 11:22:48,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 11:22:48,946 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=250} 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:48,946 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 11:22:48,947 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5cf29e8f}, purged=MutableUnsignedLongSet{size=0}} 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:22:48,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1879077423] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 11:22:48,948 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:48,948 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:48,948 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1879077423] 11:22:48,948 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:22:48,948 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 66.65 μs 11:22:48,948 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1879077423] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=0, protocol=SIMPLE}, enqueuedTicks=822278400099, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} directCommit completed 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED}: initiating purge 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 5 11:22:48,949 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b2f38e5: creating a new snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2124312498]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@111f0763 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:48,949 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2124312498]}, enqueuedTicks=822284169532} 11:22:48,949 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 11:22:48,949 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:22:48,949 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@211df26e 11:22:48,949 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=822284396897} 11:22:48,949 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 11:22:48,949 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:48,949 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 11:22:48,949 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:22:48,949 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 11:22:48,949 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,950 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@383598b5}, purged=MutableUnsignedLongSet{size=0}} 11:22:48,950 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:48,950 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,950 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1926432118] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 11:22:48,950 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:48,951 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1926432118] 11:22:48,951 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:22:48,951 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1926432118] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:48,951 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 91.34 μs 11:22:48,951 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,951 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 11:22:48,951 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 11:22:48,951 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:48,951 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 11:22:48,951 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-593: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 11:22:48,952 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-593: Message size: 395 bytes, total slices to send: 1 11:22:48,952 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-593: Message does not need to be sliced - sending original message 11:22:48,952 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 11:22:48,955 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 11:22:48,955 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2124312498], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=822284396897, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 11:22:48,955 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 11:22:48,955 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:22:48,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,955 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 11:22:48,955 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 11:22:48,955 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 11:22:48,955 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:22:48,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:48,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:48,955 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4566f9d8 11:22:48,955 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4566f9d8 11:22:48,955 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 11:22:48,955 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:22:48,955 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:48,956 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:48,956 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:22:48,956 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2734854b, splicing queue 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 11:22:48,956 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 11:22:48,956 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6724db75 queue spliced 1 messages 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 11:22:48,956 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 11:22:48,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:48,956 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: refreshing backend for shard 1 11:22:48,956 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:22:48,957 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:48,957 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:48,957 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1192963139] 11:22:48,957 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:48,957 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:22:48,957 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: replication of member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@1fe3f1d7 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 11:22:48,958 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 11:22:48,959 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} not found in inflight queue, checking pending queue 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 11:22:48,959 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2124312498]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 11:22:48,959 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#2124312498]}, enqueuedTicks=822284169532, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:22:48,959 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 11:22:48,959 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2c12a363 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:22:48,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 11:22:49,371 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 11:22:49,371 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:49,371 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 11:22:49,371 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-596: Closing 11:22:49,371 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:22:49,372 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:22:49,372 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:22:49,372 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 11:22:49,372 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:22:49,372 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 11:22:49,373 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:22:49,373 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:49,373 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:49,380 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:49,380 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:49,380 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:49,380 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:49,381 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:49,391 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 11:22:49,392 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 11:22:49,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 11:22:49,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:22:49,392 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 11:22:49,392 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:49,392 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:50,393 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-399232174813925204] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:50,412 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:50,412 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:50,413 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:50,414 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4625987186746434566] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [02/27/2024 11:22:50.415] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:50,423 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:50,423 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:50,425 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:50.426] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:50,442 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:22:50,447 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6109805698319011742] 11:22:50,447 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:50,448 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:50,448 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:50,448 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:50,449 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:50,457 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:50,457 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:50,457 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:50,460 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:50,467 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-9192591873785645449] 11:22:50,468 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:50,470 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:50,470 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:50,470 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:50,470 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:50,489 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:50,500 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3820061725250325252] 11:22:50,500 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:50,504 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:50,504 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:50,504 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:50,504 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:50,504 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:50,511 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:50,527 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:50,528 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:50,529 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 11:22:50,534 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,535 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 11:22:50,536 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 11:22:50,537 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,537 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,537 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:50,537 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,537 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,538 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 11:22:50,538 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:22:50,538 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@655700a9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7752ce07 11:22:50,538 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@736d5435 11:22:50,539 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709032970538, None)) 11:22:50,539 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032970537,0,0)) 11:22:50,539 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@78fdf363 11:22:50,539 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7752ce07 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@30285509 11:22:50,539 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 11:22:50,540 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:50,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:50,543 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 11:22:50,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:50,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,548 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,548 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,548 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227] 11:22:50,548 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,548 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 11:22:50,549 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 11:22:50,549 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 11:22:50,549 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 11:22:50,549 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,549 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#148115018] 11:22:50,549 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,549 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#1509425108 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,549 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#1522965258] 11:22:50,549 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1769406918 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,549 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#1509425108] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,553 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 11:22:50,554 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#148115018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,554 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1769406918] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,554 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#1522965258] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:50,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:50,556 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:50,556 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:50,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:22:50,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 11:22:50,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:22:50,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 11:22:50,556 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 11:22:50,556 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 11:22:50,561 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1890894490] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,570 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 11:22:50,570 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:22:50,570 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@557761eb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2fa6e408 11:22:50,571 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@277ab400 11:22:50,571 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 11:22:50,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:50,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 11:22:50,571 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:50,571 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,571 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709032970572, None)) 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309] 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 11:22:50,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032970571,0,0)) 11:22:50,572 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:22:50,572 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:22:50,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2fa6e408 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6e9720e4 11:22:50,572 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 11:22:50,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@79d0df6e 11:22:50,572 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 11:22:50,573 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 11:22:50,573 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:50,573 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#424435475] 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-726690421] 11:22:50,578 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 11:22:50,579 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 11:22:50,579 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-726690421] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,579 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1997470318] 11:22:50,579 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-1248396072 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 11:22:50,579 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1997470318] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,579 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-1248396072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,580 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:50,580 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:50,583 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:22:50,583 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 11:22:50,583 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,583 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 11:22:50,583 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 11:22:50,583 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 11:22:50,583 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:22:50,583 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 11:22:50,584 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 11:22:50,584 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#424435475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,585 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#2076873174 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:50,585 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 11:22:50,585 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#2076873174] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,585 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#858559226] 11:22:50,585 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#858559226] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:50,586 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:50,586 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:50,586 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:22:50,586 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 11:22:50,586 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 11:22:50,586 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 11:22:50,586 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 11:22:50,586 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,587 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 11:22:50,587 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 11:22:50,597 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:50,599 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:50,623 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] 11:22:50,623 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 11:22:50,623 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227] 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 11:22:50,799 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 11:22:50,799 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 11:22:50,838 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 11:22:50,838 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 11:22:51,39 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:22:51,39 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,39 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 11:22:51,108 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:22:51,108 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,108 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 11:22:51,139 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:51,159 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:51,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:22:51,328 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,328 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 11:22:51,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:22:51,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,408 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 11:22:51,459 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:51,459 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,460 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,470 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,524 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:51,548 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:22:51,548 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,548 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 11:22:51,548 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,549 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:51,550 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 11:22:51,550 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:51,550 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:51,550 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 11:22:51,550 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:51,550 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,550 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 11:22:51,551 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:51,551 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 11:22:51,553 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 11:22:51,553 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,553 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 11:22:51,553 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:22:51,553 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:51,553 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:51,553 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 11:22:51,554 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:51,629 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] 11:22:51,629 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 11:22:51,629 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309] 11:22:51,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 11:22:51,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,658 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 11:22:51,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:51,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:51,666 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 11:22:51,667 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 11:22:51,667 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 11:22:51,667 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:22:51,668 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 11:22:51,668 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,668 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 11:22:51,668 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:22:51,668 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:22:51,668 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 11:22:51,668 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 11:22:51,669 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 11:22:51,669 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:51,670 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 11:22:51,679 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6cb7e67d: creating a new transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,681 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 11:22:51,681 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:51,681 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:51,681 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] 11:22:51,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:51,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1c9378b6] 11:22:51,681 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 1 11:22:51,681 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,681 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 0 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 11:22:51,681 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 11:22:51,681 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 11:22:51,681 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] 11:22:51,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:22:51,681 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 0 11:22:51,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4bdc3c81] 11:22:51,681 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,682 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:51,682 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} allocated sequence 0 11:22:51,682 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6cb7e67d readied transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,682 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@5c7d58b0 11:22:51,682 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} scheduling timeout in 30 seconds 11:22:51,682 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@15fe5954 11:22:51,682 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:22:51,682 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1c9378b6] 11:22:51,682 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4bdc3c81] 11:22:51,683 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@61ec5ec6, attempting to connect 11:22:51,683 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard default resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6cd76c7d, attempting to connect 11:22:51,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 11:22:51,683 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 11:22:51,683 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=14210, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:22:51,683 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=11150, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:22:51,683 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:22:51,683 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:22:51,683 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@41694e7a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3ef80622 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3ef80622 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], modifications=1, protocol=THREE_PHASE} 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 0 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@15fe5954 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20e02245 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true}, enqueuedTicks=825017570549} 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6cb7e67d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 11:22:51,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5ed7f96d 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@70aff1e2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1a37329a, splicing queue 11:22:51,684 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@70aff1e2 queue spliced 0 messages 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 936.7 μs 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 0 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 11:22:51,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating canCommit 11:22:51,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:22:51,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 validated 11:22:51,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1a9a8f9c instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3be4c993 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} previous state FLUSHED 11:22:51,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed canCommit 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3be4c993 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], modifications=1, protocol=THREE_PHASE} 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 0 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@5c7d58b0 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7636d29d transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true}, enqueuedTicks=825017457556} 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6cb7e67d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6b832599 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7b927221 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1a8f8081, splicing queue 11:22:51,685 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7b927221 queue spliced 0 messages 11:22:51,686 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} in 723.2 μs 11:22:51,686 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 11:22:51,686 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true}, enqueuedTicks=825017570549, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 11:22:51,686 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating canCommit 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed canCommit 11:22:51,686 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 11:22:51,686 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642], coordinated=true}, enqueuedTicks=825017457556, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 11:22:51,686 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} canCommit completed 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 1 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@6d4b8fc0 11:22:51,686 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 11:22:51,686 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 1 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@6d4b8fc0 11:22:51,686 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:22:51,686 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7636d29d transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825021607062} 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@5b45e201 11:22:51,686 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 11:22:51,686 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:22:51,686 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@5b45e201 11:22:51,686 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:51,686 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20e02245 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825021735555} 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating preCommit 11:22:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed preCommit 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating preCommit 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 candidate ready 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - messages: [] 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 prepared 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:51,687 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed preCommit 11:22:51,687 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 11:22:51,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825021607062, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 11:22:51,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:22:51,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 11:22:51,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825021735555, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 11:22:51,688 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 2 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@3bb4edd8 11:22:51,688 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 11:22:51,688 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 2 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@3bb4edd8 11:22:51,688 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:22:51,688 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7636d29d transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825023106246} 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@ab040e1 11:22:51,688 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 11:22:51,688 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 2 11:22:51,688 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@ab040e1 11:22:51,688 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:51,688 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20e02245 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825023234139} 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating commit 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating commit 11:22:51,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 11:22:51,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 11:22:51,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 11:22:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 11:22:51,689 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:51,689 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:51,689 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,689 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 11:22:51,689 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 submitted to persistence 11:22:51,690 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 11:22:51,690 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 11:22:51,692 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 11:22:51,692 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:22:51,692 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 11:22:51,692 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:22:51,692 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:51,692 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 11:22:51,692 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 11:22:51,692 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:51,692 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 11:22:51,692 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 11:22:51,692 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:22:51,692 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:51,692 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:22:51,693 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:51,693 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 11:22:51,693 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@47fe67d5}, purged=MutableUnsignedLongSet{size=0}} 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2d6f7efa}, purged=MutableUnsignedLongSet{size=0}} 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-2103059928] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 11:22:51,694 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-195605740] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825023106246, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} doCommit completed 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1a9a8f9c completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: initiating purge 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 3 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@3f8ed2ae 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED}: initiating purge 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} allocated sequence 3 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6450527c 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:22:51,694 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7636d29d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825029655697} 11:22:51,694 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825023234139, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@41694e7a completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6e4a72ff 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED}: initiating purge 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} allocated sequence 3 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@2798d63a 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20e02245 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825029935373} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-2103059928] 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 56.97 μs 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-2103059928] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-195605740] 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@31190119, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 42.24 μs 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 11:22:51,695 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 11:22:51,695 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-195605740] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:51,695 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 11:22:51,695 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 11:22:51,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 11:22:51,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 11:22:51,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 11:22:51,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 11:22:51,696 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6cb7e67d: creating a new snapshot member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 11:22:51,696 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:22:51,696 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:51,696 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 11:22:51,696 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:22:51,697 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] and replicatedToAllIndex is -1 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=true] 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 11:22:51,697 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:22:51,697 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 11:22:51,697 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:51,697 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 11:22:51,697 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:22:51,697 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 11:22:51,697 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4e021f6e}, purged=MutableUnsignedLongSet{size=0}} 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1040056959] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 11:22:51,698 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=1, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@27ca64ab, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 11:22:51,698 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1040056959] 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:51,698 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 11:22:51,698 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 38.36 μs 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e524247 11:22:51,698 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1040056959] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e524247 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,698 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c328d54 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c328d54 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:22:51,699 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20e02245 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@3a30a882, splicing queue 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20e02245 queue spliced 1 messages 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 1 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7636d29d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1871ad49, splicing queue 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7636d29d queue spliced 1 messages 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 0 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 11:22:51,699 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 11:22:51,700 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:51,698 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@64e6751b}, purged=MutableUnsignedLongSet{size=0}} 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 11:22:51,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#392630369] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 11:22:51,700 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@23403ddc, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:22:51,700 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 11:22:51,700 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 11:22:51,700 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#392630369] 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 11:22:51,700 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 11:22:51,700 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 61.38 μs 11:22:51,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 11:22:51,700 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#392630369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:51,701 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 11:22:51,701 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 11:22:51,701 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1d2dc917, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 11:22:51,701 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 11:22:51,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 0, lastIndex: 1, isVoting: true 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 0, lastIndex: 1, isVoting: true 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:22:51,702 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:22:51,702 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:51,703 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:51,703 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:51,703 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 11:22:51,703 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 11:22:51,703 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 11:22:51,703 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 11:22:51,703 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 11:22:51,703 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@6f66a3a1 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 11:22:51,703 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825029935373, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE}: purge completed 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3ef80622 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@6487230d 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@41694e7a purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 11:22:51,704 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} not found in inflight queue, checking pending queue 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1393490642]}, enqueuedTicks=825029655697, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE}: purge completed 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3be4c993 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: purge completed 11:22:51,704 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1a9a8f9c purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 11:22:51,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:22:51,705 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 11:22:51,705 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:22:51,705 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 11:22:51,705 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 11:22:51,705 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 11:22:51,705 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 11:22:51,705 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 11:22:51,911 AM [cluster-test-shard-dispatcher-26] [WARN] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 205, lastApplied : 1, commitIndex : 1 11:22:52,108 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 11:22:52,108 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:22:52,109 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 11:22:52,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-603: Closing 11:22:52,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:22:52,109 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 11:22:52,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-604: Closing 11:22:52,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:22:52,109 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 11:22:52,110 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:22:52,110 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 11:22:52,110 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 11:22:52,110 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:22:52,110 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:22:52,110 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#836438939] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:52,111 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:52,111 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:52,117 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:52,117 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:52,117 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:52,117 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:52,119 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:52,129 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 11:22:52,129 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 11:22:52,129 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 11:22:52,129 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:22:52,129 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 11:22:52,129 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 11:22:52,129 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 11:22:52,130 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 11:22:52,130 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:52,130 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:53,133 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6109805698319011742] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:53,143 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-9192591873785645449] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:53,152 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:53,152 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:53,154 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:53.155] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:53,160 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:53,160 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:53,162 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:53.163] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:53,181 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:53,186 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4512756679635156154] 11:22:53,186 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:53,188 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:53,188 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:53,188 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:53,188 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:53,198 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:53,198 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:53,198 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:53,201 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:53,206 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1565932603910567515] 11:22:53,206 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:53,208 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:53,208 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:53,218 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:53,218 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:53,222 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:53,242 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1846371576053074074] 11:22:53,244 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:53,246 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:53,246 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:53,250 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:53,250 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:53,260 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:53,262 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:53,263 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:53,264 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 11:22:53,266 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:53,269 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1857848337] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,269 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 11:22:53,271 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 11:22:53,272 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,272 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:22:53,272 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:22:53,272 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@32c227ae to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@321ae23e 11:22:53,272 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@334d3723 11:22:53,272 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:53,272 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,272 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709032973272, None)) 11:22:53,272 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032973271,0,0)) 11:22:53,273 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@321ae23e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6ebcd6f3 11:22:53,273 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@73632e53 11:22:53,273 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 11:22:53,273 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:53,275 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:53,276 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 11:22:53,277 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:53,277 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,277 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,277 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,277 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104] 11:22:53,278 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 11:22:53,278 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 11:22:53,278 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,279 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-1069859922 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,279 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-1069859922] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,279 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-93039515] 11:22:53,280 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-93039515] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,283 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 11:22:53,283 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:53,283 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:53,285 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#344086065] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,285 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 11:22:53,285 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 11:22:53,288 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:53,288 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:53,288 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:22:53,288 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 11:22:53,288 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 11:22:53,288 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 11:22:53,289 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 11:22:53,289 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:53,289 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:53,289 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,299 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:53,305 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 11:22:53,305 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:22:53,305 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@54910604 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4693fc55 11:22:53,305 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3917ae3 11:22:53,305 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709032973305, None)) 11:22:53,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032973304,0,0)) 11:22:53,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4693fc55 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@25645701 11:22:53,306 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@250417b4 11:22:53,306 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 11:22:53,306 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:53,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:53,308 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,308 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:53,308 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1822833698] 11:22:53,308 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 11:22:53,308 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1822833698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,308 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1290867358] 11:22:53,308 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-10981380 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:53,308 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 11:22:53,308 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1290867358] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,309 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-10981380] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:53,311 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:53,311 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:53,311 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 11:22:53,311 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 11:22:53,311 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:53,311 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:22:53,311 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,311 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 11:22:53,311 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 11:22:53,312 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 11:22:53,357 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1857848337] 11:22:53,357 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 11:22:53,357 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104] 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:53,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:22:53,509 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:22:53,510 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 11:22:53,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:53,788 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:53,788 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 11:22:53,878 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:54,98 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:54,98 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:54,99 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 11:22:54,200 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:54,200 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:54,200 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:54,200 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 11:22:54,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:54,200 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:54,200 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:54,200 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,208 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:54,208 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 11:22:54,208 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,208 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:54,208 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,208 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,269 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:54,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:54,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:54,378 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 11:22:54,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:54,380 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:54,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:54,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:54,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 11:22:54,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:54,381 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:54,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:22:54,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 11:22:54,382 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:22:54,382 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:22:54,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:54,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:54,382 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 11:22:54,382 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 11:22:54,382 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 11:22:54,382 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:54,383 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 11:22:54,383 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:22:54,383 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:54,383 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:54,383 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 11:22:54,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:22:54,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:54,412 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:54,412 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 11:22:54,412 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:54,412 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:54,412 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#344086065] 11:22:54,413 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 11:22:54,413 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:54,413 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:54,413 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:54,413 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 11:22:54,413 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:54,413 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@64b28afc 11:22:54,413 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:22:54,413 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 11:22:54,414 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@257a8e37, attempting to connect 11:22:54,415 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 11:22:54,415 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=10151, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:22:54,416 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7fb0c85d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52813a6a 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52813a6a 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:22:54,417 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2263/0x00000008016c6af0@67f5809 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE}, enqueuedTicks=827748525529} 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE}} 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message size: 510 bytes, total slices to send: 1 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message does not need to be sliced - sending original message 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4b3972b9 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@52132769 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@231c6a8a, splicing queue 11:22:54,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@52132769 queue spliced 0 messages 11:22:54,419 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.787 ms 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 11:22:54,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 246 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 11:22:54,426 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=246} 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 11:22:54,427 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@48f518eb}, purged=MutableUnsignedLongSet{size=0}} 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#249389545] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 11:22:54,428 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#249389545] 11:22:54,428 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:22:54,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:54,428 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 102.2 μs 11:22:54,429 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#249389545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:54,436 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 11:22:54,436 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE}, enqueuedTicks=827748525529, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 11:22:54,436 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=4, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 11:22:54,436 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 11:22:54,436 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@4c8f0600 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED}: initiating purge 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 1 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@4ca30992 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436]}, enqueuedTicks=827771868775} 11:22:54,437 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,437 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 11:22:54,437 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:22:54,437 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@6c86e6e9 11:22:54,437 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=827772364887} 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6f28232e}, purged=MutableUnsignedLongSet{size=0}} 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1347845546] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 11:22:54,437 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1347845546] 11:22:54,438 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:22:54,438 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 11:22:54,438 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 71.28 μs 11:22:54,438 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1347845546] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:54,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 11:22:54,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 11:22:54,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:54,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 11:22:54,440 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-606: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 11:22:54,440 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-606: Message size: 392 bytes, total slices to send: 1 11:22:54,440 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-606: Message does not need to be sliced - sending original message 11:22:54,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 11:22:54,443 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 11:22:54,443 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=827772364887, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 11:22:54,443 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 11:22:54,443 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7715c17b: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:54,443 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 11:22:54,443 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:54,444 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:54,444 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1857848337] 11:22:54,444 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 11:22:54,444 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 11:22:54,444 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:54,444 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-997063607], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@7bba1d28 11:22:54,444 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:22:54,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:54,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2be2fdc4] 11:22:54,444 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2be2fdc4] 11:22:54,444 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1b7a5e82, attempting to connect 11:22:54,445 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 11:22:54,445 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=9530, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:22:54,445 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7883b5e0 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5e779445 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5e779445 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:22:54,445 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-997063607], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:22:54,446 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7715c17b finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:54,446 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 11:22:54,446 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@302833e3 11:22:54,446 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2aa49899 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4813f434, splicing queue 11:22:54,446 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2aa49899 queue spliced 0 messages 11:22:54,446 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 537.2 μs 11:22:54,446 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 11:22:54,446 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:22:54,446 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:22:54,446 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=OPEN} allocated sequence 0 11:22:54,446 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 11:22:54,446 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@1c5de953 11:22:54,446 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=1, protocol=SIMPLE}, enqueuedTicks=827781601849} 11:22:54,446 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=1, protocol=SIMPLE}} 11:22:54,447 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message size: 418 bytes, total slices to send: 1 11:22:54,447 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message does not need to be sliced - sending original message 11:22:54,447 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 ready 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 to ready 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct canCommit 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 validated 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct preCommit 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 candidate ready 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - candidate rootPath: / 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - messages: [] 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 prepared 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct commit 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 213 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 11:22:54,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 submitted to persistence 11:22:54,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 11:22:54,449 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 11:22:54,449 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 2 11:22:54,449 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:22:54,449 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 11:22:54,449 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 11:22:54,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=213} 11:22:54,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 11:22:54,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 3 11:22:54,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:22:54,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 11:22:54,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: replication of member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@f7134ba 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:54,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,451 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: commit - successfulFromPrevious: [] 11:22:54,451 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 11:22:54,451 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:22:54,451 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:22:54,451 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 11:22:54,451 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:22:54,451 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 49.71 μs 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436]}, enqueuedTicks=827771868775, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52813a6a purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7fb0c85d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=1, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=1, protocol=SIMPLE}, enqueuedTicks=827781601849, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} directCommit completed 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED}: initiating purge 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} allocated sequence 1 11:22:54,452 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@44048539 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:54,452 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 11:22:54,452 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436]}, enqueuedTicks=827787344891} 11:22:54,452 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 0 11:22:54,452 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303698@1b773e18 11:22:54,452 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=827787551796} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:54,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,453 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:22:54,453 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 11:22:54,453 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 11:22:54,453 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 48.45 μs 11:22:54,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:54,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 11:22:54,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 11:22:54,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:22:54,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 11:22:54,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 11:22:54,454 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 11:22:54,454 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=827787551796, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 11:22:54,454 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true} 11:22:54,455 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@975b387: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 11:22:54,455 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 11:22:54,455 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 0 11:22:54,455 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@1915ba1f 11:22:54,455 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=827790043223} 11:22:54,455 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 11:22:54,455 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-606: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 11:22:54,456 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-606: Message size: 369 bytes, total slices to send: 1 11:22:54,456 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-606: Message does not need to be sliced - sending original message 11:22:54,457 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 11:22:54,457 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=827790043223, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 11:22:54,457 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0} 11:22:54,457 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7715c17b: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 11:22:54,457 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 11:22:54,457 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1857848337] 11:22:54,457 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 11:22:54,457 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104] 11:22:54,457 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104] 11:22:54,508 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:22:54,508 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:54,508 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-609: Closing 11:22:54,508 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:22:54,509 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 11:22:54,509 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:54,509 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:54,514 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:54,514 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:54,514 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:54,514 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:54,516 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:54,531 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:22:54,532 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:54,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 11:22:54,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:22:54,532 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 11:22:54,532 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:54,533 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:55,529 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4512756679635156154] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:55,548 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1565932603910567515] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:55,551 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:55,551 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:55,553 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:55.555] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:55,572 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:22:55,578 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [-1554596835998830518] 11:22:55,578 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:55,580 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:55,580 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 11:22:55,580 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:55,580 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:55,590 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:55,591 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:55,593 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 11:22:55,596 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1593796034] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,598 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 11:22:55,599 AM [reinstated-member2-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:22:55,599 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 11:22:55,599 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:55,599 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 11:22:55,600 AM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 11:22:55,600 AM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4e1b7323 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@724af68f 11:22:55,600 AM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2f691901 11:22:55,600 AM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709032975600, None)) 11:22:55,600 AM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032975599,0,0)) 11:22:55,600 AM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@724af68f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@42de6975 11:22:55,600 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3491aa50 11:22:55,600 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 11:22:55,601 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:55,601 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:55,601 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 11:22:55,601 AM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 11:22:55,601 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 11:22:55,601 AM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#2109661460] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#346805153] 11:22:55,602 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1255514316 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#346805153] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1672024998] 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1255514316] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,602 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1672024998] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,603 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 11:22:55,603 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 11:22:55,603 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 11:22:55,603 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 11:22:55,603 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 11:22:55,603 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 11:22:55,603 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 246 11:22:55,603 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:55,603 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@c8bf97e}, purged=MutableUnsignedLongSet{size=0}} 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 65 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 213 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 2 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 11:22:55,604 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#-1213976520 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#-1213976520] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 2 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 1.581 ms - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:22:55,605 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 11:22:55,605 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 11:22:55,606 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 11:22:55,606 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 11:22:55,606 AM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:22:55,606 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:22:55,606 AM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 11:22:55,606 AM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 11:22:55,606 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:22:55,606 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:22:55,606 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:22:55,606 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 11:22:55,606 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 11:22:55,606 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 11:22:55,606 AM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 11:22:55,606 AM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 11:22:55,606 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:55,606 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:55,606 AM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 11:22:55,606 AM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 11:22:55,651 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1593796034] 11:22:55,651 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 11:22:55,651 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022] 11:22:55,651 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@656e18a7: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 11:22:55,651 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 11:22:55,651 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:55,651 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:55,651 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1593796034] 11:22:55,651 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=0} shard 1 11:22:55,652 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 11:22:55,652 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} allocated sequence 0 11:22:55,652 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-1108414567], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@65c23948 11:22:55,652 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} scheduling timeout in 30 seconds 11:22:55,652 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:55,652 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@48a68883] 11:22:55,652 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@48a68883] 11:22:55,652 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@63300dcf, attempting to connect 11:22:55,653 AM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: retiring state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=46768635, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 11:22:55,653 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, nanosAgo=7431, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 11:22:55,653 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:22:55,653 AM [reinstated-member2-akka.actor.default-dispatcher-13] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:22:55,653 AM [reinstated-member2-akka.actor.default-dispatcher-13] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:55,653 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4105c5c5 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fd76923 11:22:55,653 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} previous state OPEN 11:22:55,653 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fd76923 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-1108414567], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@656e18a7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6eea6318 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4bc46ebf superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5fce8895, splicing queue 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4bc46ebf queue spliced 0 messages 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 712.6 μs 11:22:55,654 AM [reinstated-member2-akka.actor.default-dispatcher-13] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0} 11:22:55,654 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@581a7aef 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@581a7aef 11:22:55,655 AM [reinstated-member2-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 11:22:55,655 AM [reinstated-member2-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@53f9af4a 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 11:22:55,655 AM [reinstated-member2-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@53f9af4a 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-614: Closing 11:22:55,655 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:22:55,656 AM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 11:22:55,656 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-613: Closing 11:22:55,656 AM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 11:22:55,656 AM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 11:22:55,656 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 11:22:55,656 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:22:55,656 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:22:55,657 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:55,657 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:22:55,657 AM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 11:22:55,657 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 11:22:55,657 AM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:22:55,657 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:55,657 AM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1593796034] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1593796034] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,658 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 11:22:55,658 AM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://reinstated-member2/user/$a#-1108414567] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#-1108414567] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,658 AM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1593796034] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1593796034] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,658 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:55,658 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:55,670 AM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:55,670 AM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:55,672 AM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:22:55.673] [reinstated-member2-akka.actor.internal-dispatcher-2] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:55,686 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:22:55,692 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-460413726702161541] 11:22:55,692 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:22:55,694 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:22:55,694 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:22:55,694 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:55,694 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:55,707 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:55,714 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:22:55,714 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:22:55,714 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:22:55,718 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1922593508212084654] 11:22:55,718 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:22:55,719 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:55,719 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:22:55,720 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:55,720 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:55,740 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:22:55,751 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4594444409611824898] 11:22:55,752 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:22:55,756 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:22:55,758 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:22:55,758 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:22:55,770 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:55,771 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:55,771 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:22:55,771 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:22:55,771 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:55,773 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,776 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,780 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-24605142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,781 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,781 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,781 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:55,781 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,785 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 11:22:55,785 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:22:55,785 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@502b90f7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@28a0b4c7 11:22:55,785 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6da22fb7 11:22:55,785 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709032975785, None)) 11:22:55,785 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032975784,0,0)) 11:22:55,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2486cae 11:22:55,785 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@28a0b4c7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3d38277c 11:22:55,785 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 11:22:55,786 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:55,786 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,786 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:55,787 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:22:55,788 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,789 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314] 11:22:55,789 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 11:22:55,789 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,790 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1597123958] 11:22:55,790 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1345460009 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,790 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1345460009] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,790 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 11:22:55,790 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1597123958] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,791 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,792 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#712317498] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:55,792 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:55,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:55,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:22:55,792 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 11:22:55,792 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 11:22:55,792 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 11:22:55,792 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,792 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 11:22:55,793 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,793 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 11:22:55,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,793 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:22:55,793 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@30d5efbb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5aab1415 11:22:55,793 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:22:55,793 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,794 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1fa99124 11:22:55,794 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709032975794, None)) 11:22:55,794 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032975793,0,0)) 11:22:55,794 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5aab1415 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2cf5ea60 11:22:55,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@464350bc 11:22:55,794 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 11:22:55,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:22:55,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,794 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1700502236] 11:22:55,795 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 11:22:55,795 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 11:22:55,796 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1700502236] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,796 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-964602110 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,796 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#2023417635] 11:22:55,796 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-964602110] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,796 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#2023417635] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:22:55,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:22:55,797 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:22:55,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 11:22:55,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 11:22:55,798 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:55,798 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:22:55,798 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 11:22:55,798 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,798 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:55,798 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 11:22:55,799 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:22:55,802 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:22:55,844 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-24605142] 11:22:55,845 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 11:22:55,845 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314] 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:22:56,103 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 11:22:56,103 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 11:22:56,325 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:22:56,393 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:56,393 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,393 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 11:22:56,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:56,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,673 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 11:22:56,714 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:22:56,714 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:22:56,714 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:56,714 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,714 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,714 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:56,714 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,715 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,715 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,724 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:22:56,724 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,724 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,724 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:22:56,724 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,724 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,782 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:22:56,972 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:56,972 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,972 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 11:22:56,973 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:22:56,976 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:22:56,976 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:22:56,976 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 11:22:56,976 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:56,976 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:22:56,976 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:22:56,977 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 11:22:56,977 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 11:22:56,977 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 11:22:56,977 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:56,977 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:56,977 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 11:22:56,977 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 11:22:56,978 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:22:56,978 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:56,978 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 11:22:56,978 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:56,978 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:22:56,978 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 11:22:56,978 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 11:22:56,979 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 11:22:56,979 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:22:57,0 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@261655dd: creating a new snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 11:22:57,0 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 11:22:57,0 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:57,0 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:57,0 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#712317498] 11:22:57,0 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=0} shard 1 11:22:57,1 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 11:22:57,1 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:57,1 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@2f8acd81 11:22:57,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:57,1 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} scheduling timeout in 2 seconds 11:22:57,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 11:22:57,1 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 11:22:57,1 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@43e84005, attempting to connect 11:22:57,5 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} is not yet known 11:22:57,5 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=11440, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:22:57,6 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@75d5af57 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@512fe35b 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@512fe35b 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@7a383fd7 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1993572353 nanoseconds 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c1b118d transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=830335983633} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@261655dd finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6200166e 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3f605d4b superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@41d1bf7, splicing queue 11:22:57,7 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3f605d4b queue spliced 0 messages 11:22:57,8 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 897.1 μs 11:22:57,8 AM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: allocating new open snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 11:22:57,8 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-616: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 11:22:57,8 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-616: Message size: 239 bytes, total slices to send: 1 11:22:57,8 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-616: Message does not need to be sliced - sending original message 11:22:57,13 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 11:22:57,13 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=830335983633, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 11:22:57,13 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0} 11:22:57,13 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:22:57,13 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,13 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-619: Closing 11:22:57,14 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 11:22:57,14 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,14 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:57,14 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:22:57,19 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:57,19 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:57,19 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:57,19 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:22:57,22 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:22:57,34 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 11:22:57,134 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,135 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@261655dd: creating a new transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 11:22:57,135 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 11:22:57,135 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:22:57,135 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@261655dd readied transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 11:22:57,135 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@1282e48c 11:22:57,135 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:57,135 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c1b118d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], modifications=1, protocol=SIMPLE}, enqueuedTicks=830470304439} 11:22:57,135 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-617: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], modifications=1, protocol=SIMPLE}} 11:22:57,135 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,135 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-617: Message size: 401 bytes, total slices to send: 1 11:22:57,135 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-617: Message does not need to be sliced - sending original message 11:22:57,135 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:22:57,135 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,135 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 11:22:57,135 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 11:22:57,135 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:22:57,326 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 11:22:57,326 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 11:22:57,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:22:57,327 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c1b118d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@101ee580, splicing queue 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1808109391 nanoseconds 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c1b118d queue spliced 1 messages 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: refreshing backend for shard 1 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:57,327 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#712317498] 11:22:57,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:57,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 11:22:57,495 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:57,495 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,496 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 11:22:57,496 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,546 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 11:22:57,546 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 11:22:57,546 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:57,547 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:57,547 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:57,547 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:57,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:57,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=7, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,696 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 7 11:22:57,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:57,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=8, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:57,837 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 8 11:22:57,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,26 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-460413726702161541] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:58,32 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-460413726702161541] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:22:58,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:58,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=9, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,55 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 9 11:22:58,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,216 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:58,216 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=10, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,216 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 10 11:22:58,216 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:58,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=11, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,385 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 11 11:22:58,385 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,565 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:58,565 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:22:58,565 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:22:58,565 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#712317498] 11:22:58,566 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:22:58,566 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 11:22:58,575 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:58,575 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=12, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,575 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 12 11:22:58,575 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,726 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:58,726 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=13, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,726 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 13 11:22:58,726 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 11:22:58,785 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 11:22:58,786 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:22:58,786 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:58,786 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:58,786 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:22:58,886 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:58,886 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=14, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:58,886 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 14 11:22:58,886 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:59,16 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 11:22:59,16 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} is empty 11:22:59,16 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 11:22:59,26 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:59,26 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 11:22:59,26 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:59,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 11:22:59,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=15, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:59,96 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 15 11:22:59,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:59,155 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:59,155 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#999408869], modifications=1, protocol=SIMPLE}, enqueuedTicks=830470304439, sessionId=0, txSequence=1} 11:22:59,157 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 11:22:59,158 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:22:59,161 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} directCommit completed 11:22:59,161 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED}: initiating purge 11:22:59,161 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 11:22:59,161 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#999408869]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@52c3099e 11:22:59,161 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 11:22:59,161 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999952169 nanoseconds 11:22:59,161 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:22:59,162 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:59,162 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:22:59,162 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 11:22:59,162 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:22:59,162 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:22:59,163 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 11:22:59,163 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 11:22:59,163 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 11:22:59,163 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 11:22:59,164 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:22:59,164 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:00,178 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:00,179 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:23:00.181] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:00,182 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1922593508212084654] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:00,199 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:00,199 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:00,201 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:00.203] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:00,219 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:00,224 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2831653964982583224] 11:23:00,224 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:00,226 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:00,226 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:00,226 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:00,226 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:00,237 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:00,237 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:00,237 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:00,239 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:00,248 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1981631027046969607] 11:23:00,249 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:00,250 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:00,250 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:00,257 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:00,257 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:00,264 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:00,272 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8301301424850662291] 11:23:00,272 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:00,272 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:00,272 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:00,279 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:00,280 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:00,282 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:00,282 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:00,283 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 11:23:00,285 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1687002139] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,286 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:23:00,292 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 11:23:00,292 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 11:23:00,292 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:23:00,292 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,292 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@8409717 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a46c3f0 11:23:00,292 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1f4456dc 11:23:00,292 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,292 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:00,292 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,292 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,292 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709032980292, None)) 11:23:00,292 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032980291,0,0)) 11:23:00,292 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a46c3f0 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@74da45e6 11:23:00,292 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 11:23:00,293 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@461ca84e 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574] 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418] 11:23:00,295 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 11:23:00,295 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:23:00,295 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 11:23:00,295 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:23:00,295 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,295 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,295 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#788549280] 11:23:00,296 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:00,296 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1523906588] 11:23:00,296 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-19553841] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,297 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 11:23:00,298 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-19553841 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,298 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:00,298 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:00,298 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 11:23:00,298 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 11:23:00,298 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:00,298 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:00,299 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1984291188 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,299 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:00,299 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#788549280] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,299 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1523906588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,299 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1984291188] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,300 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:00,300 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:23:00,301 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:23:00,301 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-973278546] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,301 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 11:23:00,301 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 11:23:00,301 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:23:00,301 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:00,301 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 11:23:00,308 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:23:00,309 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:00,309 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:23:00,309 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:23:00,309 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:23:00,309 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 11:23:00,310 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 11:23:00,310 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:23:00,310 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5aab4327 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@30ef2435 11:23:00,310 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@34d0a6f0 11:23:00,310 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709032980310, None)) 11:23:00,310 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 11:23:00,311 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,311 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,311 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:00,311 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,311 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,311 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032980309,0,0)) 11:23:00,311 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@30ef2435 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@40e9faed 11:23:00,311 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 11:23:00,311 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5d5f9806 11:23:00,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:00,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,312 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-585560850] 11:23:00,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,313 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-1221494349] 11:23:00,313 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 11:23:00,313 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 11:23:00,313 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:23:00,314 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 11:23:00,314 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-210991763 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,314 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1325296096] 11:23:00,314 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1894724965 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,314 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-585560850] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,315 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:00,315 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:00,315 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:00,315 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:00,315 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:23:00,315 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:23:00,316 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,316 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 11:23:00,316 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 11:23:00,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:00,316 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:00,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:00,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:23:00,316 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:23:00,316 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 11:23:00,316 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:00,317 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1632842298] 11:23:00,317 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-1221494349] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,317 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-210991763] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,317 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1325296096] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,317 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1894724965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,317 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1632842298] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:00,319 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 11:23:00,319 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 11:23:00,319 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 11:23:00,319 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,319 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 11:23:00,319 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 11:23:00,362 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1687002139] 11:23:00,362 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 11:23:00,362 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574] 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:23:00,597 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 11:23:00,597 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 11:23:00,606 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 11:23:00,606 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 11:23:00,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:23:00,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:00,846 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 11:23:00,887 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:23:00,887 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:00,887 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 11:23:01,12 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:01,12 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:01,146 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:23:01,146 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,146 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 11:23:01,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:23:01,197 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,197 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 11:23:01,237 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:01,237 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,238 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,266 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:01,266 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,278 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,387 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:23:01,387 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,387 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 11:23:01,387 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 11:23:01,390 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:01,390 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 11:23:01,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 11:23:01,394 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,394 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 11:23:01,395 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:23:01,395 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:01,396 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:23:01,396 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:01,396 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:01,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 11:23:01,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,477 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 11:23:01,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,479 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 11:23:01,480 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 11:23:01,480 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 11:23:01,481 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 11:23:01,481 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,481 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 11:23:01,481 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 11:23:01,481 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:01,481 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:01,481 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 11:23:01,483 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:23:01,483 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:01,509 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 11:23:01,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1687002139] 11:23:01,567 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 11:23:01,567 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418] 11:23:01,568 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@45712b20 11:23:01,568 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 1218945601, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@1cfa186e} 11:23:01,568 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@45712b20 changed state from IDLE to TX_OPEN 11:23:01,568 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 11:23:01,568 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 11:23:01,568 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 11:23:01,568 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-973278546] 11:23:01,569 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:01,569 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 11:23:01,569 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:23:01,569 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:01,569 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:23:01,569 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 11:23:01,569 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:01,569 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4650d3b, attempting to connect 11:23:01,569 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 11:23:01,569 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 11:23:01,569 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 11:23:01,569 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-973278546] 11:23:01,569 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 11:23:01,569 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 11:23:01,569 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 11:23:01,569 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 11:23:01,570 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@2f27cac1 11:23:01,570 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:01,570 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@55639d97 11:23:01,570 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:23:01,570 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 11:23:01,570 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@45712b20 changed state from TX_OPEN to IDLE 11:23:01,570 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@45712b20 readied transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 11:23:01,570 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 11:23:01,570 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 11:23:01,570 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@1afd141f 11:23:01,570 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 11:23:01,570 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@23782ff1 11:23:01,570 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@63b19a0a, attempting to connect 11:23:01,570 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 11:23:01,570 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=16491, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:23:01,571 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 11:23:01,571 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=9280, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:23:01,571 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:01,571 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7bd3a5e6 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@54e9c3c8 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c6964fc transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834904100793} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@54e9c3c8 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2224/0x00000008016a8238@62bc2435 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c6964fc transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY}, enqueuedTicks=834904934112} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY}} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message size: 407 bytes, total slices to send: 1 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message does not need to be sliced - sending original message 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@2230b692 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c6964fc transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=834905181128} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE}} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message size: 299 bytes, total slices to send: 1 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message does not need to be sliced - sending original message 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@45712b20 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6a60cc0b 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@695d972f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@50b8ceb7, splicing queue 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@695d972f queue spliced 0 messages 11:23:01,572 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} in 829.3 μs 11:23:01,573 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@58c2e276 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7ed54ca 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20c4ff7 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834904735858} 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7ed54ca 11:23:01,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2224/0x00000008016a8238@71da7b2d 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20c4ff7 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY}, enqueuedTicks=834904987083} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY}} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message size: 492 bytes, total slices to send: 1 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message does not need to be sliced - sending original message 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@5af6bbb8 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20c4ff7 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=834905244069} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE}} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message size: 300 bytes, total slices to send: 1 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-623: Message does not need to be sliced - sending original message 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@45712b20 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1e80bd55 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3d91d400 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@538915ca, splicing queue 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3d91d400 queue spliced 0 messages 11:23:01,574 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} in 1.140 ms 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7d1b8da0 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 ready 11:23:01,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transitioned member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Validating transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 validated 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 11:23:01,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 11:23:01,582 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:23:01,582 AM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 11:23:01,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 11:23:01,582 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 11:23:01,582 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 11:23:01,582 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@29b3b725 11:23:01,582 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 11:23:01,582 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:01,582 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY}, enqueuedTicks=834904934112, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 11:23:01,582 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 11:23:01,582 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 11:23:01,583 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=834905181128, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 11:23:01,583 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 11:23:01,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 11:23:01,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:01,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 11:23:01,589 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=1, protocol=READY} failed to process}} 11:23:01,589 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY}, enqueuedTicks=834904987083, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=1, protocol=READY} failed to process}} 11:23:01,589 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=1, protocol=READY} failed to process} 11:23:01,589 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 11:23:01,589 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 11:23:01,589 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1108950118], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=834905244069, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 11:23:01,589 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@693a041a failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1127) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.resolveResult(VotingFuture.java:91) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.voteNo(VotingFuture.java:67) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$canCommit$5(AbstractProxyTransaction.java:549) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$17(RemoteProxyTransaction.java:516) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013086a0@7e48ba7e 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013086a0@7e48ba7e 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c6964fc transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834925621128} 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013086a0@1eeaa9e7 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x00000008013086a0@1eeaa9e7 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20c4ff7 transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834925751451} 11:23:01,590 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 11:23:01,593 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborting 11:23:01,593 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=3, message=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1108950118]}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:01,594 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: aborting head of queue member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 in state member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,594 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1: abort - successfulFromPrevious: [] 11:23:01,594 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborted 11:23:01,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,597 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 11:23:01,598 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 11:23:01,598 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834925751451, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 11:23:01,598 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} abort completed 11:23:01,598 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@19664531 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 3 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@22468729 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20c4ff7 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834933849307} 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834925621128, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [ERROR] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during Abort. org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} abort completed 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@2e32cd74 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 3 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@385b6854 11:23:01,599 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c6964fc transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834934355318} 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 11:23:01,600 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2226/0x00000008016a8a48@65327d48 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4115b868}, purged=MutableUnsignedLongSet{size=0}} 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:23:01,601 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 11:23:01,602 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 11:23:01,602 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 11:23:01,602 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:01,602 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@693a041a is no longer running 11:23:01,602 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:23:01,602 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 11:23:01,602 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834904735858, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 11:23:01,602 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58f760c0 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58f760c0 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:01,603 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,603 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:01,603 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:01,603 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}]]] 11:23:01,603 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 11:23:01,603 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 11:23:01,603 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@23d8b54c 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@23d8b54c 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}]]] 11:23:01,604 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:01,604 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:01,604 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 11:23:01,604 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@26e2d43}, purged=MutableUnsignedLongSet{size=0}} 11:23:01,604 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 11:23:01,604 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 11:23:01,605 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}]]] 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:01,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78} 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}]]] 11:23:01,605 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 2 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:01,605 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 11:23:01,605 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c6964fc superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@14676047, splicing queue 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 11:23:01,605 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:01,605 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:23:01,605 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c6964fc queue spliced 2 messages 11:23:01,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 11:23:01,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 1 11:23:01,606 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-513884418], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 11:23:01,606 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 11:23:01,606 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 11:23:01,606 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-973278546] 11:23:01,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:01,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:01,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 11:23:01,606 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 11:23:01,606 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 11:23:01,606 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 11:23:01,607 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:01,607 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20c4ff7 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@7864e0af, splicing queue 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@20c4ff7 queue spliced 1 messages 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 0, lastIndex: 1, isVoting: true 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 2 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:23:01,607 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-973278546] 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78} 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@2acd7c2d 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: finished purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:01,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2226/0x00000008016a8a48@7ace609b 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3ff6fc51}, purged=MutableUnsignedLongSet{size=0}} 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@1539fa79 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: finished purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 11:23:01,608 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:01,609 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:01,609 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:01,609 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 11:23:01,609 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 11:23:01,609 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 11:23:01,609 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:01,609 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 11:23:01,609 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 11:23:01,609 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 11:23:01,610 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 11:23:01,610 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78} 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@33a6ae19}, purged=MutableUnsignedLongSet{size=0}} 11:23:01,610 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 11:23:01,610 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:01,610 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:01,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834933849307, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE}: purge completed 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7ed54ca purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: purge completed 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@58c2e276 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834904100793, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1108950118]}, enqueuedTicks=834934355318, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@54e9c3c8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 11:23:01,611 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7bd3a5e6 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:23:02,17 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:23:02,17 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-265207574]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:02,17 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:02,17 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-627: Closing 11:23:02,17 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 11:23:02,18 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:02,18 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-626: Closing 11:23:02,18 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 11:23:02,18 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:23:02,18 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:23:02,18 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 11:23:02,18 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 11:23:02,18 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:23:02,18 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:23:02,18 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#530733605] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#530733605] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:02,19 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1687002139] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1687002139] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:02,19 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1687002139] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1687002139] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:02,19 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:02,19 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:02,24 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:02,24 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:02,24 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:02,24 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:02,26 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:02,39 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 11:23:02,39 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 11:23:02,39 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:23:02,39 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 11:23:02,39 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 11:23:02,39 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 11:23:02,39 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 11:23:02,40 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 11:23:02,40 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:02,40 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:02,314 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:03,33 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2831653964982583224] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:03,53 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1981631027046969607] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:03,58 AM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:03,58 AM [cluster-test-akka.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:03,60 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:03.061] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:03,70 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:03,71 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:03,72 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:03.073] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:03,91 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:23:03,101 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6241052054034292860] 11:23:03,101 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:03,102 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:03,102 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:03,102 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:03,102 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:03,114 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:03,114 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:03,114 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:03,116 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:03,120 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [9201018822716294477] 11:23:03,121 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:03,129 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:03,129 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:03,129 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:03,129 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:03,140 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:03,155 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7473106252603195923] 11:23:03,155 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:03,157 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:03,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:03,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:03,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:03,168 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:03,171 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:03,172 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 11:23:03,180 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1486888369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,182 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 11:23:03,183 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 11:23:03,183 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:03,183 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2fd30a7a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2acd0ea1 11:23:03,184 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 11:23:03,184 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,184 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,184 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:03,184 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,184 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,184 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6fcf959c 11:23:03,185 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:03,185 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709032983185, None)) 11:23:03,186 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032983184,0,0)) 11:23:03,186 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2acd0ea1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@711a05d 11:23:03,187 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@20e061d7 11:23:03,187 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 11:23:03,187 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:03,188 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:03,190 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 11:23:03,191 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:03,191 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,191 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,191 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,192 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 11:23:03,192 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,192 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#893198066 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,193 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 11:23:03,193 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#893198066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,206 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2c12a363 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2124312498], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@5906ee46 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-651775506], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:03,206 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0} 11:23:03,206 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3be4c993 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@92b82d0 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#638919309], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 11:23:03,206 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3ef80622 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1393490642]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@5cd624a0 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-4221227], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:03,206 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0} 11:23:03,206 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52813a6a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 11:23:03,206 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@42a52a8f 11:23:03,206 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT}, enqueuedTicks=836541777958} 11:23:03,206 AM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT}} 11:23:03,207 AM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message size: 287 bytes, total slices to send: 1 11:23:03,207 AM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message does not need to be sliced - sending original message 11:23:03,207 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0} 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7883b5e0 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-997063607], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@6bfd693e 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5e779445 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-997063607]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@343e42c3 11:23:03,207 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52813a6a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 1 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@5553f383 11:23:03,207 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT}, enqueuedTicks=836542317321} 11:23:03,207 AM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT}} 11:23:03,207 AM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message size: 287 bytes, total slices to send: 1 11:23:03,207 AM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message does not need to be sliced - sending original message 11:23:03,207 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0} 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52813a6a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 1 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@2211c8da 11:23:03,207 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5df660c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT}, enqueuedTicks=836542506625} 11:23:03,207 AM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1725992436], modifications=0, protocol=ABORT}} 11:23:03,207 AM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message size: 287 bytes, total slices to send: 1 11:23:03,207 AM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-607: Message does not need to be sliced - sending original message 11:23:03,207 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0} 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5e779445 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-997063607]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@76f39d9e 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#878792104], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4105c5c5 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=1, replyTo=Actor[akka://reinstated-member2/user/$a#-1108414567], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@5f94d109 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fd76923 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-1108414567]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@48eb43 11:23:03,207 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-91801022], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:03,207 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@75d5af57 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:23:03,208 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:03,208 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#999408869], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@63ba7084 11:23:03,208 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} has a successor, not scheduling timer 11:23:03,208 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:23:03,208 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#999408869], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013038d0@94f1e00 11:23:03,208 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#250367314], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 11:23:03,208 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0} 11:23:03,191 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136] 11:23:03,208 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,208 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,208 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,208 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272] 11:23:03,208 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 11:23:03,209 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 11:23:03,209 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,209 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#521738503] 11:23:03,209 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#521738503] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,210 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:03,212 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:03,214 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:03,214 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 11:23:03,214 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 11:23:03,218 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-246330521] 11:23:03,218 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-246330521] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,218 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#222305990] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,219 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 11:23:03,220 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#1035869095 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,220 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 11:23:03,220 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:23:03,220 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#1035869095] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,220 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 11:23:03,226 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:03,226 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:23:03,227 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 11:23:03,227 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:23:03,227 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 11:23:03,230 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 11:23:03,230 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 11:23:03,230 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:03,230 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:03,230 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@fd99413 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43e561c8 11:23:03,230 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@47daed6c 11:23:03,230 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 11:23:03,230 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709032983230, None)) 11:23:03,230 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:03,230 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,230 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,230 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032983229,0,0)) 11:23:03,231 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@238abaa5 11:23:03,231 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43e561c8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3390ce59 11:23:03,231 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560] 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:03,232 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 11:23:03,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895] 11:23:03,232 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,233 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 11:23:03,233 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 11:23:03,233 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 11:23:03,233 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-613186302] 11:23:03,233 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#711872632 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:03,234 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-1825053115 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 11:23:03,235 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,235 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:03,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:03,235 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:03,235 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:03,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 11:23:03,235 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:03,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 11:23:03,235 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:23:03,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:03,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:23:03,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 11:23:03,235 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 11:23:03,235 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,235 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 11:23:03,235 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 11:23:03,235 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,235 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:23:03,235 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#578408961] 11:23:03,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 11:23:03,235 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:23:03,235 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 11:23:03,236 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#711872632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,236 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-613186302] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,236 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-1825053115] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,236 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#578408961] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:03,282 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1486888369] 11:23:03,282 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 11:23:03,282 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136] 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:23:03,438 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:23:03,438 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:23:03,448 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:23:03,448 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 11:23:03,668 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:23:03,668 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,668 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 11:23:03,678 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:23:03,678 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,678 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 11:23:03,907 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:23:03,907 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:03,908 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 11:23:03,928 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:23:03,928 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:23:03,928 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 11:23:04,118 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:04,118 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,118 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,119 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:04,119 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,124 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,176 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:04,208 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:23:04,208 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,208 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 11:23:04,208 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,208 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:04,210 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:23:04,211 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 11:23:04,212 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,212 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 11:23:04,212 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,213 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,213 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:04,213 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:04,213 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:23:04,213 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:23:04,213 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,215 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:23:04,215 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,215 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:04,215 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:04,215 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 11:23:04,218 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 11:23:04,218 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,218 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 11:23:04,218 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,218 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,219 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:23:04,220 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 11:23:04,220 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,220 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:04,220 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:04,220 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:23:04,220 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 11:23:04,220 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:23:04,221 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 11:23:04,221 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,221 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,221 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,221 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:04,221 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:04,221 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:23:04,221 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 11:23:04,221 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:23:04,221 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,287 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1486888369] 11:23:04,287 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 11:23:04,288 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272] 11:23:04,288 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:04,290 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:04,293 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 11:23:04,296 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1526483104] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,296 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,299 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:04,299 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,299 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,300 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 11:23:04,300 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,300 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7f3118cb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@784774bc 11:23:04,300 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7c78ff6c 11:23:04,300 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709032984300, None)) 11:23:04,300 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032984299,0,0)) 11:23:04,300 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@61669fa7 11:23:04,300 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@784774bc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@36362b9a 11:23:04,300 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,301 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#2087263876] 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#2087263876] 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#2087263876] 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1475622225] 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,301 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1475622225] 11:23:04,301 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,301 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1475622225] 11:23:04,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#222305990] 11:23:04,301 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 11:23:04,301 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:23:04,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:04,301 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 11:23:04,301 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#2087263876] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,301 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 11:23:04,301 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:04,301 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:04,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#222305990] 11:23:04,302 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 2 11:23:04,302 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:23:04,302 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:04,302 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#2129722345 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,302 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#2129722345] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,302 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:23:04,302 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1764517101] 11:23:04,302 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:23:04,302 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 11:23:04,302 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:23:04,302 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:04,302 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 11:23:04,302 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:23:04,302 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 11:23:04,302 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 11:23:04,302 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:04,302 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:23:04,302 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:04,302 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 11:23:04,302 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 11:23:04,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1486888369] 11:23:04,303 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#-438131607 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,303 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-135054349] 11:23:04,303 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1764517101] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,304 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:23:04,304 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 11:23:04,304 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@43f44f6a, attempting to connect 11:23:04,304 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3a03d7af, attempting to connect 11:23:04,305 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1475622225] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,305 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#-438131607] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 11:23:04,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 11:23:04,306 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 11:23:04,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=8160, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,306 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=7970, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,306 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:23:04,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:23:04,307 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:04,307 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136] 11:23:04,307 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:04,307 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@718777a1 11:23:04,307 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:04,307 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@7f1d9952 11:23:04,307 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1355d542 skipping transactions [1] 11:23:04,307 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} scheduling timeout in 30 seconds 11:23:04,307 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@59b1b701 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@341c087 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} previous state SEALED 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@341c087 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@2e1b3a7 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@774f2c8f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE}, enqueuedTicks=837642536416} 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-631: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE}} 11:23:04,308 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 11:23:04,308 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-631: Message size: 402 bytes, total slices to send: 1 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-631: Message does not need to be sliced - sending original message 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@341c087 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 11:23:04,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 11:23:04,308 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ad44fa8 11:23:04,308 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ad44fa8 11:23:04,308 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 11:23:04,308 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,308 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,309 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:23:04,309 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:04,309 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4fe7bba1 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@424dde0c superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6cdf0314, splicing queue 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@424dde0c queue spliced 0 messages 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.345 ms 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1355d542 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5485076 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5485076 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@3d8a22f1 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c47fe23 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=837642749871} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-631: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1, protocol=THREE_PHASE}} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-631: Message size: 391 bytes, total slices to send: 1 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-631: Message does not need to be sliced - sending original message 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 11:23:04,309 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@47676012 11:23:04,310 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@46c808b2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6ab29845, splicing queue 11:23:04,310 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@46c808b2 queue spliced 0 messages 11:23:04,310 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 523.8 μs 11:23:04,310 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,311 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 11:23:04,311 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,311 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 11:23:04,311 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:23:04,311 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 11:23:04,311 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 11:23:04,311 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:23:04,311 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 11:23:04,311 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,312 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 11:23:04,312 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:23:04,312 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:04,312 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@774f2c8f superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@446e43b5, splicing queue 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@774f2c8f queue spliced 1 messages 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 1 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:04,312 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#222305990] 11:23:04,312 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:04,312 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,313 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,313 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:04,313 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 11:23:04,313 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 11:23:04,313 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:23:04,313 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:23:04,313 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: -1, lastIndex: -1, isVoting: true 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: allocating new open transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 ready 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: transitioned member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 to ready 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating canCommit 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,315 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: request Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 validated 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:04,315 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed canCommit 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 11:23:04,317 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 11:23:04,317 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 11:23:04,317 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 11:23:04,317 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 11:23:04,317 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=837642749871, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 11:23:04,317 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-634: Closing 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 11:23:04,323 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 11:23:04,323 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 11:23:04,323 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:23:04,323 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:23:04,323 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 11:23:04,323 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 11:23:04,324 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 11:23:04,324 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5c9a7610] 11:23:04,324 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:23:04,324 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5c9a7610] 11:23:04,324 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:23:04,324 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 11:23:04,324 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5b7bee83, attempting to connect 11:23:04,324 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 11:23:04,324 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=7630, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,324 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 11:23:04,325 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,325 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,325 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:04,325 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:04,325 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 16.33 ms 11:23:04,325 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:04,325 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 11:23:04,325 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:23:04,325 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@341c087 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40f009d4 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state SEALED 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} previous state SEALED 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40f009d4 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=2, protocol=THREE_PHASE} 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 0 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@2e1b3a7 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:04,325 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2871a3c2 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=true}, enqueuedTicks=837642536416} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40f009d4 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], modifications=1} 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating canCommit 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 validated 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed canCommit 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@56bc36f4 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7376f052 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@666145b4, splicing queue 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7376f052 queue spliced 0 messages 11:23:04,326 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 948.9 μs 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=true} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 11:23:04,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=true}, enqueuedTicks=837642536416, sessionId=2, txSequence=0} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 11:23:04,326 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@6869c894 11:23:04,326 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@22198047 11:23:04,326 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,326 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,326 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@22198047 11:23:04,326 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:04,326 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2871a3c2 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837661565203} 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@315d5a5f 11:23:04,326 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating preCommit 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 1 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,326 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@7c618f8f 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 candidate ready 11:23:04,326 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:23:04,326 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c47fe23 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837661703086} 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - messages: [] 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 prepared 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:04,326 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed preCommit 11:23:04,327 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:23:04,327 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 11:23:04,327 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:04,327 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837661565203, sessionId=2, txSequence=1} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 11:23:04,327 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:04,327 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating preCommit 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 candidate ready 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - candidate rootPath: / 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - messages: [] 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 prepared 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:04,327 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed preCommit 11:23:04,328 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 11:23:04,328 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837661703086, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 11:23:04,328 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@349f6d4d 11:23:04,328 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@349f6d4d 11:23:04,328 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,328 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,328 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 2 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@349f6d4d 11:23:04,328 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:04,328 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2871a3c2 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837663313603} 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@6f4b4623 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating commit 11:23:04,328 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 2 11:23:04,328 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@6f4b4623 11:23:04,328 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:04,328 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c47fe23 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837663499087} 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-1-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 11:23:04,328 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 submitted to persistence 11:23:04,329 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating commit 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,330 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 submitted to persistence 11:23:04,330 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,331 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:04,331 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:04,332 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@68dbe361}, purged=MutableUnsignedLongSet{size=0}} 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: commit - successfulFromPrevious: [] 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#706530935] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 11:23:04,332 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#706530935] 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,332 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:23:04,332 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 123.1 μs 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#706530935] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,333 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 11:23:04,333 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837663499087, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 11:23:04,333 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 11:23:04,333 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 11:23:04,333 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1355d542 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@33f89161 11:23:04,333 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED}: initiating purge 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} allocated sequence 3 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@4c41a7c 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:04,333 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c47fe23 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837668653276} 11:23:04,334 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2afad846}, purged=MutableUnsignedLongSet{size=0}} 11:23:04,334 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 11:23:04,334 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,334 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1291887711] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 11:23:04,334 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1291887711] 11:23:04,334 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:23:04,334 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:23:04,334 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 58.06 μs 11:23:04,334 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1291887711] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 4 - updating term 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:04,334 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:04,335 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:23:04,335 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 11:23:04,335 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 11:23:04,335 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 11:23:04,335 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 11:23:04,335 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,335 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 11:23:04,335 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 11:23:04,336 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@22a9209e}, purged=MutableUnsignedLongSet{size=0}} 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 11:23:04,337 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,338 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1367407994] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 11:23:04,338 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1367407994] 11:23:04,338 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:23:04,338 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 53.94 μs 11:23:04,338 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 11:23:04,338 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,338 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,338 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:04,338 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 11:23:04,338 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:23:04,338 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,338 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 11:23:04,338 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 11:23:04,338 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:04,338 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,338 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 11:23:04,339 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 11:23:04,339 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:23:04,339 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:04,339 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=170} 11:23:04,339 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 11:23:04,339 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 11:23:04,339 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 11:23:04,339 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@d9a91cd}, purged=MutableUnsignedLongSet{size=0}} 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1287530524] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:04,340 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1287530524] 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} with Envelope{sessionId=2, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837663313603, sessionId=2, txSequence=2} with Envelope{sessionId=2, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@59b1b701 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6ce3afed 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 79.36 μs 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@29e13c15 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-883456136], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED}: initiating purge 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} allocated sequence 3 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1287530524] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@11663a98 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:04,340 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2871a3c2 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837675451532} 11:23:04,340 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 11:23:04,340 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@13ef5a94 11:23:04,340 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2871a3c2 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=false}, enqueuedTicks=837675686888} 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct canCommit 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 validated 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct preCommit 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 candidate ready 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - candidate rootPath: / 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - messages: [] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 prepared 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct commit 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-1-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}], SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 11:23:04,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 submitted to persistence 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@3a4b55a4 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 11:23:04,342 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 11:23:04,342 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 11:23:04,342 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:04,342 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 11:23:04,342 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 11:23:04,342 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 11:23:04,342 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 11:23:04,342 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 11:23:04,342 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,343 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@16ea1d5a}, purged=MutableUnsignedLongSet{size=0}} 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1748365452] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 11:23:04,343 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1748365452] 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 11:23:04,343 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=229} 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 11:23:04,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:04,343 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 92.37 μs 11:23:04,344 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 11:23:04,344 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 11:23:04,344 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@b05c8e2 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: commit - successfulFromPrevious: [] 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837668653276, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE}: purge completed 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5485076 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1355d542 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} with Envelope{sessionId=2, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837675451532, sessionId=2, txSequence=3} with Envelope{sessionId=2, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 11:23:04,344 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE}: purge completed 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40f009d4 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@341c087 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@59b1b701 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=false} with Envelope{sessionId=2, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997], coordinated=false}, enqueuedTicks=837675686888, sessionId=2, txSequence=4} with Envelope{sessionId=2, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} directCommit completed 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED}: initiating purge 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} allocated sequence 1 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@7fe33607 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 56.51 μs 11:23:04,344 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2871a3c2 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837679723690} 11:23:04,345 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,345 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e: creating a new snapshot member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 11:23:04,345 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 11:23:04,345 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 11:23:04,345 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:04,345 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 11:23:04,345 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5b28ecb3: creating a new snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:23:04,345 AM [main] [DEBUG] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 11:23:04,345 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:04,345 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:04,345 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1526483104] 11:23:04,345 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 11:23:04,345 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 11:23:04,345 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:04,345 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@38ac4f16 11:23:04,345 AM [main] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 11:23:04,345 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 11:23:04,345 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 11:23:04,346 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4eb9821a, attempting to connect 11:23:04,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,346 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 11:23:04,346 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 11:23:04,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 11:23:04,346 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 132.7 μs 11:23:04,347 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,347 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 11:23:04,347 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 11:23:04,347 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=9701, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,347 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 11:23:04,347 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:23:04,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:23:04,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 11:23:04,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 11:23:04,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 4 11:23:04,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:04,348 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:04,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@48789ed9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4061f843 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4061f843 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@109b0b75 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1409d183 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=837680488228} 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5b28ecb3 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5b6590a1 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@49a5de66 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1ada484, splicing queue 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@49a5de66 queue spliced 0 messages 11:23:04,348 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 509.3 μs 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 3 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@3a02802b 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:23:04,349 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 11:23:04,349 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} with Envelope{sessionId=2, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 11:23:04,349 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=837679723690, sessionId=2, txSequence=5} with Envelope{sessionId=2, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 11:23:04,349 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE}: purge completed 11:23:04,349 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40f009d4 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} 11:23:04,350 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new open snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 11:23:04,350 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-633: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 11:23:04,350 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-633: Message size: 363 bytes, total slices to send: 1 11:23:04,350 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-633: Message does not need to be sliced - sending original message 11:23:04,351 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 11:23:04,351 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=837680488228, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 11:23:04,351 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0} 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 3 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 3 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 11:23:04,352 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,352 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,353 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 11:23:04,353 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 11:23:04,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 11:23:04,353 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,353 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:23:04,353 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:23:04,353 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 11:23:04,353 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 11:23:04,353 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:23:04,353 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 11:23:04,353 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 11:23:04,353 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:23:04,354 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,354 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 11:23:04,354 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:23:04,354 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 11:23:04,354 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,354 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 11:23:04,354 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@27b155e5 11:23:04,354 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@27b155e5 11:23:04,354 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 11:23:04,354 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 11:23:04,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 11:23:04,356 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 11:23:04,356 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:04,356 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 11:23:04,356 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c47fe23 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@180213dc, splicing queue 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4c47fe23 queue spliced 0 messages 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 2 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:04,356 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#222305990] 11:23:04,356 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:04,356 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:04,356 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 11:23:04,356 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,356 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 11:23:04,356 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 11:23:04,356 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,356 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,356 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 11:23:04,357 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 11:23:04,357 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:23:04,357 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,357 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 11:23:04,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-3-shard-people-testLeadershipTransferOnShutdown 11:23:04,358 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:23:04,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:23:04,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:04,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:23:04,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 11:23:04,358 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 11:23:04,358 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:23:04,358 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 11:23:04,358 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 11:23:04,358 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-635: Closing 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 11:23:04,360 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 11:23:04,360 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 11:23:04,361 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 11:23:04,361 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6438404c] 11:23:04,361 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:23:04,361 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6438404c] 11:23:04,361 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4ea182cd, attempting to connect 11:23:04,361 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 11:23:04,361 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,362 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 11:23:04,362 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 7.416 ms 11:23:04,362 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 11:23:04,362 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:04,362 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:04,362 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 11:23:04,362 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 11:23:04,362 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:04,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 11:23:04,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 11:23:04,363 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 11:23:04,363 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5485076 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@66b90afa 11:23:04,363 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19493f0e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@8db1880 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6c670312 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7d1af9b1, splicing queue 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6c670312 queue spliced 0 messages 11:23:04,363 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 11:23:04,363 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1606809272], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1419832895], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 239.8 μs 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 11:23:04,363 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 11:23:04,363 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:23:04,363 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 11:23:04,363 AM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:23:04,363 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:23:04,364 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:04,364 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:04,367 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:04,367 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:04,367 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:04,367 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:04,369 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:04,378 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 11:23:04,379 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 11:23:04,379 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-639: Closing 11:23:04,379 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:23:04,379 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 11:23:04,379 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-640: Closing 11:23:04,379 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 11:23:04,379 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 11:23:04,380 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:04,380 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:05,375 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6241052054034292860] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:05,395 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [9201018822716294477] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:05,396 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:05,396 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:05,398 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:05.399] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:05,411 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:05,411 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:05,413 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:05.414] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:05,445 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:23:05,452 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5129841248150708060] 11:23:05,452 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:05,464 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:05,464 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:05,464 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:05,464 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:05,478 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:05,478 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:05,478 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:05,486 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:05,511 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7305867194535420156] 11:23:05,511 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:05,513 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:05,513 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:05,513 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:05,513 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:05,527 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:05,537 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5483556835715003439] 11:23:05,537 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:05,540 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:05,540 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:05,540 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:05,540 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:05,545 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:05,547 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:05,550 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:05,551 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:05,552 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 11:23:05,557 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,562 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:23:05,562 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 11:23:05,562 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:05,563 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@b076601 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3129752c 11:23:05,563 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1df07131 11:23:05,563 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 11:23:05,563 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,563 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,563 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709032985563, None)) 11:23:05,563 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:05,563 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,563 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,563 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032985562,0,0)) 11:23:05,563 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3129752c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2c2d3dbf 11:23:05,564 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 11:23:05,564 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:05,564 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@387cffca 11:23:05,564 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:05,564 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,565 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,565 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,565 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:05,565 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,565 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:23:05,566 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,566 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:23:05,566 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#1955379520 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,566 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:23:05,566 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:23:05,566 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#1955379520] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,566 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:05,566 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,566 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,566 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] 11:23:05,567 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,568 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#158477012 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,568 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#158477012] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,568 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1353596581] 11:23:05,568 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1353596581] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,568 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#796526249] 11:23:05,568 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#796526249] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,569 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 11:23:05,571 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:05,573 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:05,573 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:05,573 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:05,573 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:05,573 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 11:23:05,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 11:23:05,573 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:23:05,573 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:23:05,573 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:23:05,574 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:23:05,574 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:23:05,574 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:23:05,574 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 11:23:05,574 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:05,574 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1a9d1263 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@552712ef 11:23:05,575 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2020e7a9 11:23:05,575 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709032985575, None)) 11:23:05,575 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032985574,0,0)) 11:23:05,575 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@552712ef to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7de61266 11:23:05,575 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 11:23:05,575 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 11:23:05,575 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,575 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,576 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:05,576 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,576 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,576 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@20ce856d 11:23:05,576 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:05,576 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,586 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:05,588 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902] 11:23:05,593 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:23:05,594 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:23:05,594 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 11:23:05,594 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-126476977] 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#1235859695 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-359121016 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1180570928] 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#1235859695] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-126476977] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-359121016] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,595 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1180570928] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:05,596 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:05,596 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:05,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 11:23:05,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:05,598 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:05,598 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:05,598 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,598 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 11:23:05,598 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 11:23:05,598 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 11:23:05,598 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:23:05,598 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 11:23:05,599 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 11:23:05,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:05,626 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:05,626 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:05,804 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:05,805 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:23:05,805 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:05,875 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:23:05,875 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:23:06,64 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:23:06,64 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,64 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 11:23:06,188 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:06,188 AM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:06,194 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:23:06,194 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,194 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 11:23:06,384 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:23:06,384 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,384 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 11:23:06,475 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:06,475 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,475 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,485 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,486 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:23:06,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,514 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 11:23:06,515 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=null 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 11:23:06,517 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:23:06,517 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:06,518 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:06,518 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:06,518 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:23:06,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:06,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,518 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:06,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:06,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:06,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 11:23:06,519 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 11:23:06,519 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:06,556 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:06,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 11:23:06,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,624 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 11:23:06,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,625 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 11:23:06,627 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:23:06,627 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:23:06,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:06,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,628 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 11:23:06,628 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:06,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:06,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:06,628 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:23:06,632 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 11:23:06,632 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:06,634 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 11:23:06,681 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:06,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 11:23:06,681 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] 11:23:06,682 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:06,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:06,682 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:06,682 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:06,682 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:06,682 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:06,682 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,682 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,682 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:23:06,683 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 11:23:06,683 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:06,683 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:06,683 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:06,683 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 11:23:06,683 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:23:06,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:06,683 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 11:23:06,683 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:06,683 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:06,683 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:06,683 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 11:23:06,683 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:23:06,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:06,683 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:23:06,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@aefbe2d 11:23:06,684 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:06,684 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@4ba9e60d 11:23:06,684 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 11:23:06,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:06,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:06,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:06,684 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:06,684 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:06,684 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:06,685 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@17071e30, attempting to connect 11:23:06,685 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7b0d3b59, attempting to connect 11:23:06,686 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 11:23:06,687 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=16260, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:06,687 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 11:23:06,687 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=15890, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:06,688 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:06,688 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@764004db instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@74e65ade 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840018891246} 11:23:06,688 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 395 bytes, total slices to send: 1 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@67c84f40 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1d7fa62f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2e3aad38, splicing queue 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1d7fa62f queue spliced 0 messages 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 780.7 μs 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@461de7d4 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@48816cc8 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840019045650} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 400 bytes, total slices to send: 1 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2fe34c20 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@34abf803 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4e92efee, splicing queue 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@34abf803 queue spliced 0 messages 11:23:06,689 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 522.4 μs 11:23:06,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,691 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,691 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 ready 11:23:06,691 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 ready 11:23:06,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 to ready 11:23:06,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating canCommit 11:23:06,691 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 to ready 11:23:06,691 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating canCommit 11:23:06,691 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,691 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,692 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 validated 11:23:06,692 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:06,692 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 validated 11:23:06,692 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:06,692 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed canCommit 11:23:06,692 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed canCommit 11:23:06,692 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 11:23:06,692 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840018891246, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 11:23:06,692 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 11:23:06,692 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 11:23:06,692 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840019045650, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 11:23:06,692 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 11:23:06,692 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:06,692 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@38dce76a 11:23:06,692 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 11:23:06,692 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:23:06,692 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@4f35388a 11:23:06,692 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:06,693 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840027848462} 11:23:06,693 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 11:23:06,693 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@4ba826ef 11:23:06,693 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 11:23:06,693 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 11:23:06,693 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@352a80a2 11:23:06,693 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:06,693 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840028055547} 11:23:06,693 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating preCommit 11:23:06,693 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,693 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating preCommit 11:23:06,693 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,693 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 candidate ready 11:23:06,693 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 candidate ready 11:23:06,694 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - candidate rootPath: / 11:23:06,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:23:06,694 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - messages: [] 11:23:06,694 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 11:23:06,694 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 prepared 11:23:06,694 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:06,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - messages: [] 11:23:06,694 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed preCommit 11:23:06,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:06,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 prepared 11:23:06,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:06,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed preCommit 11:23:06,695 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 11:23:06,695 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840028055547, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 11:23:06,695 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:06,695 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 11:23:06,695 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840027848462, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 11:23:06,695 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@c85426a 11:23:06,695 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 2 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@c85426a 11:23:06,695 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:06,695 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840030381471} 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@610824b 11:23:06,695 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 2 11:23:06,695 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@610824b 11:23:06,695 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:06,695 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840030516464} 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating commit 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating commit 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=184}] 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=184}] 11:23:06,696 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:06,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,697 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 submitted to persistence 11:23:06,697 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=184}]]] 11:23:06,697 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 submitted to persistence 11:23:06,697 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:06,697 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=184}] 11:23:06,698 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=184}]]] 11:23:06,698 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:06,698 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:06,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 11:23:06,698 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 11:23:06,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:06,698 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=184}] 11:23:06,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:06,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=180} 11:23:06,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:06,698 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 11:23:06,698 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:06,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 11:23:06,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@17b1a7f7}, purged=MutableUnsignedLongSet{size=0}} 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@47cd8b2c}, purged=MutableUnsignedLongSet{size=0}} 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: commit - successfulFromPrevious: [] 11:23:06,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:23:06,699 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:06,700 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1836167883] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:23:06,700 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1690474256] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:23:06,700 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:06,700 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1690474256] 11:23:06,700 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:06,700 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1836167883] 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 74.92 μs 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1690474256] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1836167883] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:06,700 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 130.6 μs 11:23:06,701 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 11:23:06,701 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840030516464, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 11:23:06,701 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 11:23:06,701 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@461de7d4 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 11:23:06,701 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 11:23:06,701 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@2dccfbc2 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED}: initiating purge 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} allocated sequence 3 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@68ce701f 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840036815028} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840030381471, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@764004db completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@59ecb9a5 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED}: initiating purge 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} allocated sequence 3 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@13b45bb1 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:06,702 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840037350671} 11:23:06,702 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=184}] 11:23:06,703 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:23:06,703 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:06,703 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 11:23:06,703 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 11:23:06,703 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,703 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 11:23:06,703 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@51d21f27}, purged=MutableUnsignedLongSet{size=0}} 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@25acff23}, purged=MutableUnsignedLongSet{size=0}} 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:06,703 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,703 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,704 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-129072860] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:23:06,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-577248153] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-129072860] 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-577248153] 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:06,704 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:23:06,704 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 99.32 μs 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-129072860] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 75.92 μs 11:23:06,704 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-577248153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:06,704 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,705 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,705 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 11:23:06,705 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 11:23:06,705 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:06,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 11:23:06,705 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:06,706 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 11:23:06,706 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:23:06,706 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@3c3f39fd 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 3 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:06,707 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 3 11:23:06,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:23:06,708 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 11:23:06,708 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840036815028, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 11:23:06,708 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE}: purge completed 11:23:06,708 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} 11:23:06,708 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 11:23:06,708 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@461de7d4 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 11:23:06,755 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,805 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,855 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,864 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,864 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 11:23:06,865 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 11:23:06,865 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:06,865 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:06,865 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 11:23:06,865 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 11:23:06,865 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:23:06,865 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:06,866 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 11:23:06,867 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@36b75e25 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:06,868 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:23:06,869 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 11:23:06,869 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840037350671, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 11:23:06,869 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE}: purge completed 11:23:06,869 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} 11:23:06,869 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:06,869 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@764004db purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 11:23:06,869 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 11:23:06,906 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,906 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:06,906 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 11:23:06,906 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902] 11:23:06,906 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902] 11:23:06,906 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:23:06,906 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:23:06,906 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:06,907 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:23:06,907 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} allocated sequence 0 11:23:06,907 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 11:23:06,907 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@29131a0b 11:23:06,907 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:06,907 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840242189378} 11:23:06,907 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} 11:23:06,907 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 406 bytes, total slices to send: 1 11:23:06,907 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:06,907 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@1e3e8beb 11:23:06,907 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:06,907 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840242599698} 11:23:06,907 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} 11:23:06,907 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 402 bytes, total slices to send: 1 11:23:06,907 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 ready 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 to ready 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 ready 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 to ready 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 11:23:06,911 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:06,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 11:23:06,912 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 11:23:06,912 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840242599698, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 11:23:06,912 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} canCommit completed 11:23:06,912 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 11:23:06,912 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840242189378, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 11:23:06,912 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} canCommit completed 11:23:06,912 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:23:06,913 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:23:06,913 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} allocated sequence 0 11:23:06,913 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:23:06,913 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} allocated sequence 0 11:23:06,913 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 11:23:06,913 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:06,913 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:06,913 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:06,913 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:06,913 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:06,913 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:06,913 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,913 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,914 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:23:06,914 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:23:06,914 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 0 11:23:06,914 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@54e28e32 11:23:06,914 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:06,914 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2}, enqueuedTicks=840249271121} 11:23:06,914 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2}} 11:23:06,914 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 490 bytes, total slices to send: 1 11:23:06,914 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:06,914 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 1 11:23:06,915 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@7c01af8e 11:23:06,915 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2}, enqueuedTicks=840249916066} 11:23:06,915 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2}} 11:23:06,915 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 491 bytes, total slices to send: 1 11:23:06,915 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:06,915 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 2 11:23:06,915 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:23:06,915 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:23:06,915 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} allocated sequence 0 11:23:06,915 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:23:06,915 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:23:06,915 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} allocated sequence 0 11:23:06,915 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:06,915 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:06,915 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:06,916 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:06,916 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:06,917 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} with Envelope{sessionId=0, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 11:23:06,917 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2}, enqueuedTicks=840249271121, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 11:23:06,917 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0} 11:23:06,917 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 11:23:06,917 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2}, enqueuedTicks=840249916066, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 11:23:06,917 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1} 11:23:06,966 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:06,966 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:06,966 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:06,966 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,966 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:06,967 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:06,967 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-647: Closing 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:23:06,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:06,967 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-648: Closing 11:23:06,968 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:06,968 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 11:23:06,968 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:06,968 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:23:06,968 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 11:23:06,968 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:06,968 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 11:23:07,17 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:07,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:07,17 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:07,18 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@1705d63c 11:23:07,18 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:07,18 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840353067137} 11:23:07,18 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} 11:23:07,18 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 452 bytes, total slices to send: 1 11:23:07,18 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:07,18 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@73486b61 11:23:07,18 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:07,18 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=840353503487} 11:23:07,18 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, message: Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} 11:23:07,18 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message size: 435 bytes, total slices to send: 1 11:23:07,18 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-644: Message does not need to be sliced - sending original message 11:23:07,18 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:23:07,18 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 11:23:07,19 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:23:07,19 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 11:23:07,19 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:23:07,19 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 11:23:07,19 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 11:23:07,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:07,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:07,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:23:07,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:07,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:23:07,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:07,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:07,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:07,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 11:23:07,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 11:23:07,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:07,20 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:07,21 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 11:23:07,21 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 11:23:07,21 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:07,21 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1115384444], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@575eb94, splicing queue 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60e43952 queue spliced 1 messages 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 1 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:07,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5de71fb2, splicing queue 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f0b15f1 queue spliced 1 messages 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 2 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:07,27 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:07,28 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:07,28 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:07,28 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:07,28 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:07,28 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:07,28 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:07,28 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1985fbff, attempting to connect 11:23:07,28 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4ac97a98, attempting to connect 11:23:07,30 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 11:23:07,30 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 11:23:07,30 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry$d], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 11:23:07,30 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$c], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 11:23:07,42 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Connect attempt to people failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:07,42 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:07,43 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 2 shard people failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:07,43 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:07,43 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 2, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:07,43 AM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:07,43 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:07,44 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:07,69 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:07,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:07,69 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:07,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 11:23:07,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:07,70 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:07,70 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:23:07,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:07,71 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:23:07,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:23:07,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:23:07,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 11:23:07,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:07,73 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:23:07,73 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:23:07,73 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:23:07,73 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:23:07,73 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:07,73 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:07,73 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:23:07,73 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:07,73 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:23:07,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:07,74 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:07,74 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:07,74 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:07,74 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:07,74 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 11:23:07,74 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:23:07,74 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:07,75 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 11:23:07,119 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 11:23:07,120 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902] 11:23:07,120 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} allocated sequence 1 11:23:07,120 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@1c2ca465 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 11:23:07,120 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:07,120 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} allocated sequence 1 11:23:07,120 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@283a0ec6 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 4 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 11:23:07,120 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 skipping transactions [3, 4, 5] 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:07,120 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 11:23:07,120 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:07,120 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:07,120 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 11:23:07,120 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:23:07,121 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:07,121 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 11:23:07,121 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=3] with higher term appears reachable, updating term. 11:23:07,121 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=3]" message is greater than follower's term 3 - updating term 11:23:07,121 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 11:23:07,121 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 3 11:23:07,122 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:07,122 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:07,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:07,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:07,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 11:23:07,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:07,123 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:07,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 11:23:07,123 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:07,123 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:07,123 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 11:23:07,123 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:23:07,123 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 11:23:07,124 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 11:23:07,124 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:23:07,124 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:07,124 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:07,124 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 11:23:07,124 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:07,124 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:07,124 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 11:23:07,124 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 11:23:07,129 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 11:23:07,129 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:07,130 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 11:23:07,189 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 113, lastApplied : 1, commitIndex : 1 11:23:07,239 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 108, lastApplied : 1, commitIndex : 1 11:23:07,309 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:07,359 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:07,429 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:07,480 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 11:23:07,495 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:23:07,550 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:07,599 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:07,669 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 117, lastApplied : 1, commitIndex : 1 11:23:07,720 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 11:23:07,789 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:07,839 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 117, lastApplied : 1, commitIndex : 1 11:23:07,909 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:07,959 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:08,28 AM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 11:23:08,57 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,57 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:08,58 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,58 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2410be11] 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:08,58 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:08,58 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@191c0ee6] 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2410be11] 11:23:08,58 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@191c0ee6] 11:23:08,58 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@694149dc, attempting to connect 11:23:08,58 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@615dc68d, attempting to connect 11:23:08,59 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:08,59 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4912ac65 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} previous state FLUSHED 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} previous state FLUSHED 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4912ac65 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 0 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1247/0x0000000801308ae0@5a9f68a3 11:23:08,59 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} scheduling timeout in 30 seconds 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353503487} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 replayed 1 successful requests 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 1 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@283a0ec6 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840455703956} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4912ac65 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating preCommit 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 0 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@73486b61 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353503487} 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 candidate ready 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - candidate rootPath: / 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1cae0995 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7b8a9e41 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@11872a94, splicing queue 11:23:08,60 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7b8a9e41 queue spliced 0 messages 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - messages: [] 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: preCommit - successfulFromPrevious: [] 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 prepared 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:08,60 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed preCommit 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 1.492 ms 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating canCommit 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 validated 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed canCommit 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} previous state FLUSHED 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} previous state SEALED 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} previous state SEALED 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} previous state SEALED 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 0 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1247/0x0000000801308ae0@5a9f68a3 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353067137} 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 replayed 1 successful requests 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 1 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@1c2ca465 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840455155563} 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,61 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1, protocol=THREE_PHASE} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating preCommit 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 0 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@1705d63c 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353067137} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 candidate ready 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - candidate rootPath: / 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - messages: [] 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 prepared 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed preCommit 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=2} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 replayed 2 successful requests 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating canCommit 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1} 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 validated 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:08,62 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed canCommit 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], modifications=1} 11:23:08,62 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4d63584d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@146cd64b 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@53bc632b superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@447114bb, splicing queue 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@53bc632b queue spliced 0 messages 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.132 ms 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353503487, sessionId=2, txSequence=0} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840455703956, sessionId=2, txSequence=1} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353503487, sessionId=2, txSequence=2} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} canCommit completed 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353067137, sessionId=3, txSequence=0} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=840455155563, sessionId=3, txSequence=1} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=true}, enqueuedTicks=840353067137, sessionId=3, txSequence=2} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 11:23:08,63 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} canCommit completed 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 2 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@20aa4a8c 11:23:08,64 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,64 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,64 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 2 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@20aa4a8c 11:23:08,64 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,64 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841399332790} 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 2 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@59dcdd70 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating commit 11:23:08,64 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,64 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,64 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 2 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 78 11:23:08,64 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@59dcdd70 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 11:23:08,64 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 11:23:08,64 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841399584526} 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 11:23:08,64 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 submitted to persistence 11:23:08,64 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating commit 11:23:08,65 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,65 AM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 185 11:23:08,65 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=185}] 11:23:08,65 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=185}] 11:23:08,65 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:08,65 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 11:23:08,65 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:23:08,65 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 submitted to persistence 11:23:08,66 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=185}] 11:23:08,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 11:23:08,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:08,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:23:08,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=78} 11:23:08,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 11:23:08,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:23:08,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,67 AM [cluster-test-shard-dispatcher-19] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 106, lastApplied : 1, commitIndex : 1 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: commit - successfulFromPrevious: [] 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 11:23:08,67 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:08,67 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=185}]]] 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841399332790, sessionId=3, txSequence=3} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} doCommit completed 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 3 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@34f884a0 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,67 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED}: initiating purge 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} allocated sequence 3 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@46d1c2f 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 28.09 μs 11:23:08,67 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841402626306} 11:23:08,68 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,68 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 11:23:08,68 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 11:23:08,68 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:08,68 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 11:23:08,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=185}]]] 11:23:08,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 11:23:08,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:23:08,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=185} 11:23:08,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=185}] 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:23:08,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 11:23:08,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,68 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 11:23:08,68 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,69 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 27.73 μs 11:23:08,69 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=185}] 11:23:08,69 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 11:23:08,69 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}]]] 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: commit - successfulFromPrevious: [] 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841399584526, sessionId=2, txSequence=3} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} doCommit completed 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: initiating purge 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 3 11:23:08,69 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 4 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@28a49850 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,69 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED}: initiating purge 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} allocated sequence 3 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@5889d460 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841404790566} 11:23:08,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,70 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 76.79 μs 11:23:08,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 11:23:08,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 11:23:08,70 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 1 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@69b72af3 11:23:08,70 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,70 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,70 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 1 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@69b72af3 11:23:08,70 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841405586784} 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 1 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@1bbbf363 11:23:08,70 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,70 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,70 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 1 11:23:08,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@1bbbf363 11:23:08,70 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841405705577} 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating preCommit 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating preCommit 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 candidate ready 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 candidate ready 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - candidate rootPath: / 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - candidate rootPath: / 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - messages: [] 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - messages: [] 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: preCommit - successfulFromPrevious: [] 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 prepared 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 prepared 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:08,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed preCommit 11:23:08,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed preCommit 11:23:08,71 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 11:23:08,71 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841405586784, sessionId=3, txSequence=5} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 11:23:08,71 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:08,71 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 11:23:08,71 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841405705577, sessionId=2, txSequence=5} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 11:23:08,71 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:08,71 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 2 11:23:08,71 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@401a7f30 11:23:08,71 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,71 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,71 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,71 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 2 11:23:08,71 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@401a7f30 11:23:08,72 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841406867454} 11:23:08,72 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 2 11:23:08,72 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@102a3572 11:23:08,72 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,72 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,72 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,72 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 2 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating commit 11:23:08,72 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@102a3572 11:23:08,72 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841407015847} 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating commit 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 223 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=223}] 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=223}] 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=223}]]] 11:23:08,72 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 submitted to persistence 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 submitted to persistence 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}]]] 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 4 11:23:08,73 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 11:23:08,73 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,73 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 44.69 μs 11:23:08,74 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 11:23:08,74 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 11:23:08,74 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 11:23:08,74 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=223}] 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=223}]]] 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=223} 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=223}] 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 3, nextIndex: 4 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@347b6063 11:23:08,75 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=223}] 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 4 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 11:23:08,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:08,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 11:23:08,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 11:23:08,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 11:23:08,75 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 11:23:08,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 11:23:08,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:23:08,75 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841402626306, sessionId=3, txSequence=4} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 11:23:08,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 11:23:08,75 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE}: purge completed 11:23:08,75 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} 11:23:08,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@44a4d0ce 11:23:08,75 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 11:23:08,76 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,76 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 11:23:08,76 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841404790566, sessionId=2, txSequence=4} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,76 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE}: purge completed 11:23:08,76 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4912ac65 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} 11:23:08,76 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: purge completed 11:23:08,76 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: commit - successfulFromPrevious: [] 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 4 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:23:08,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841407015847, sessionId=2, txSequence=6} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} doCommit completed 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: initiating purge 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 3 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@3469373a 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#59712802], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED}: initiating purge 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} allocated sequence 3 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 84.47 μs 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@545acece 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 11:23:08,77 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6a800ce8 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841411966911} 11:23:08,77 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,77 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 11:23:08,77 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 11:23:08,77 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:08,78 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 3 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 4 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 11:23:08,83 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,83 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 4 11:23:08,83 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 11:23:08,84 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 54.88 μs 11:23:08,84 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:08,84 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 11:23:08,84 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 11:23:08,85 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: commit - successfulFromPrevious: [] 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841406867454, sessionId=3, txSequence=6} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 30.00 μs 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} doCommit completed 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: initiating purge 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 3 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@4e8c5faf 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED}: initiating purge 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} allocated sequence 3 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@332171a3 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,85 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841420251131} 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 11:23:08,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:08,85 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 11:23:08,86 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} allocated sequence 0 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 11:23:08,86 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@6c94560d 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:23:08,86 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false}, enqueuedTicks=841420933347} 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct canCommit 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 validated 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct preCommit 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 candidate ready 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - candidate rootPath: / 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 5, nextIndex: 6 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - messages: [] 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: preCommit - successfulFromPrevious: [] 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@14ad82c3 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 prepared 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct commit 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 4 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:23:08,86 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 11:23:08,86 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 325 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 11:23:08,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 11:23:08,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 submitted to persistence 11:23:08,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 11:23:08,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 67.06 μs 11:23:08,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841411966911, sessionId=2, txSequence=7} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE}: purge completed 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4912ac65 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: purge completed 11:23:08,87 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5307de79 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 11:23:08,87 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 11:23:08,87 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 11:23:08,88 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 5 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 4 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 11:23:08,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=325} 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 11:23:08,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@25ff5256 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841420251131, sessionId=3, txSequence=7} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE}: purge completed 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: commit - successfulFromPrevious: [] 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: purge completed 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7ed08586 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false} with Envelope{sessionId=3, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false}, enqueuedTicks=841420933347, sessionId=3, txSequence=8} with Envelope{sessionId=3, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} directCommit completed 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED}: initiating purge 11:23:08,89 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} allocated sequence 1 11:23:08,89 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 11:23:08,90 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@5071a063 11:23:08,90 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 75.09 μs 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:23:08,90 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,90 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841424823616} 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 11:23:08,90 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} allocated sequence 0 11:23:08,90 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@1d2ec644 11:23:08,90 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false}, enqueuedTicks=841425137873} 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct canCommit 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 validated 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct preCommit 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 candidate ready 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - candidate rootPath: / 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - messages: [] 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: preCommit - successfulFromPrevious: [] 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 prepared 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct commit 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 8, leaderSnapShotIndex: 5 11:23:08,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,91 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 11:23:08,91 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 submitted to persistence 11:23:08,97 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 11:23:08,97 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 11:23:08,99 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=8, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,99 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 51.58 μs 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 11:23:08,99 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 8, nextIndex: 9 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 6 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@2eacf441 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 8 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 11:23:08,101 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 11:23:08,101 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841424823616, sessionId=3, txSequence=9} with Envelope{sessionId=3, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE}: purge completed 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: commit - successfulFromPrevious: [] 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from COMMIT_PENDING to COMMITTED 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false} with Envelope{sessionId=3, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false}, enqueuedTicks=841425137873, sessionId=3, txSequence=10} with Envelope{sessionId=3, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} directCommit completed 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED}: initiating purge 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} allocated sequence 1 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@1dc9ed29 11:23:08,102 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} allocated sequence 0 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,102 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@131dbeba 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841437249622} 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 11:23:08,102 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 68.31 μs 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 11:23:08,102 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false}, enqueuedTicks=841437389175} 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct canCommit 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,102 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 validated 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct preCommit 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 candidate ready 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - candidate rootPath: / 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - messages: [] 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: preCommit - successfulFromPrevious: [] 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 prepared 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct commit 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 10, leaderSnapShotIndex: 7 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 11:23:08,103 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 submitted to persistence 11:23:08,104 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 11:23:08,104 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 8 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 6, lastIndex(): 8 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 7 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 8 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,105 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 11:23:08,106 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=10, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,106 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,106 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 87.76 μs 11:23:08,107 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 11:23:08,107 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 11:23:08,107 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 11:23:08,107 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 10, nextIndex: 11 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 8 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 9 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@4992abcd 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 10 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841437249622, sessionId=3, txSequence=11} with Envelope{sessionId=3, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE}: purge completed 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: commit - successfulFromPrevious: [] 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false} with Envelope{sessionId=3, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1115384444], coordinated=false}, enqueuedTicks=841437389175, sessionId=3, txSequence=12} with Envelope{sessionId=3, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} directCommit completed 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED}: initiating purge 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} allocated sequence 1 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@318ff5b7 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d21ece6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841444505919} 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 11:23:08,109 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 44.88 μs 11:23:08,109 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 11:23:08,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 11:23:08,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:08,110 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:08,110 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:08,110 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:08,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 11:23:08,110 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#94104108] 11:23:08,110 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#558642584] 11:23:08,110 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 10 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 8, lastIndex(): 10 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 9 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 10 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,111 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 11:23:08,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 11:23:08,111 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 90.81 μs 11:23:08,112 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 11:23:08,112 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 11 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,112 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 11:23:08,112 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:08,112 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153] 11:23:08,112 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@47a13e24: creating a new snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:23:08,112 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 11:23:08,112 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:08,112 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:08,112 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:08,112 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:08,113 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 11:23:08,113 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:23:08,113 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:08,113 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@164de108 11:23:08,113 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:08,113 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:08,113 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5e51f093, attempting to connect 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 11:23:08,113 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=11, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 11:23:08,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=11190, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 11, nextIndex: 12 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 11 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@3a9a3351 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 11:23:08,114 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 11:23:08,115 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]} with Envelope{sessionId=3, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1115384444]}, enqueuedTicks=841444505919, sessionId=3, txSequence=13} with Envelope{sessionId=3, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE}: purge completed 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4e784c94 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} 11:23:08,115 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@367426a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6def6166 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6def6166 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:08,115 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@1ba9ffb8 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60997265 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=841448056650} 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@47a13e24 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@76ebcb41 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@b60b084 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5fe0da6f, splicing queue 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@b60b084 queue spliced 0 messages 11:23:08,116 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 527.5 μs 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 11 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 11 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 11:23:08,116 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 11:23:08,117 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 11:23:08,117 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 11:23:08,117 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 514 bytes, total slices to send: 1 11:23:08,117 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 11:23:08,119 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 11:23:08,119 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=841448056650, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 11:23:08,119 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0} 11:23:08,119 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 11:23:08,119 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:08,119 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:08,119 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] 11:23:08,119 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:08,119 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 11:23:08,119 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 11:23:08,119 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:08,119 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@5d7c0fe5 11:23:08,119 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:08,119 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 11:23:08,119 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 11:23:08,120 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2d2782c, attempting to connect 11:23:08,121 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 11:23:08,121 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=5451, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:08,122 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:08,122 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:23:08,122 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,122 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@45fde8e8 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3d9a27a9 11:23:08,122 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state OPEN 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3d9a27a9 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@53b64251 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad5bfd8 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=841454755544} 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@47a13e24 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@269e6e44 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@60ba5a97 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1ff281b, splicing queue 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@60ba5a97 queue spliced 0 messages 11:23:08,123 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 837.9 μs 11:23:08,124 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 11:23:08,124 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-646: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 11:23:08,125 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-646: Message size: 360 bytes, total slices to send: 1 11:23:08,125 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-646: Message does not need to be sliced - sending original message 11:23:08,125 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 11:23:08,125 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=841454755544, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 11:23:08,126 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0} 11:23:08,126 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:08,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:08,126 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 11:23:08,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 11:23:08,126 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 11:23:08,131 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:08,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:23:08,131 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:08,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 11:23:08,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:23:08,131 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 11:23:08,131 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:23:08,132 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:08,132 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 11:23:08,132 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:08,132 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 11:23:08,132 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:08,133 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1548742582] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:08,134 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:08,134 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:08,139 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:08,139 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:08,139 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:08,139 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:08,157 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 11:23:08,158 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 11:23:08,158 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 11:23:08,158 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-649: Closing 11:23:08,158 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-650: Closing 11:23:08,158 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:23:08,158 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 11:23:08,158 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 11:23:08,159 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:08,159 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:09,147 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5129841248150708060] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:09,176 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7305867194535420156] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:09,179 AM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:09,179 AM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:09,181 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:09.183] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:09,189 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:09,189 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:09,191 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:09.192] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:09,203 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:09,208 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3663035708762118169] 11:23:09,208 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:09,210 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:09,210 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:09,210 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:09,210 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:09,218 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:09,218 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:09,218 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:09,230 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:09,235 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2949720795770270794] 11:23:09,236 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:09,237 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:09,237 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:09,237 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:09,237 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:09,262 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:09,272 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1459130032822189877] 11:23:09,272 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:09,275 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:09,275 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:09,275 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:09,284 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:09,284 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:09,285 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:09,287 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:09,288 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,296 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#905929556] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,296 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,298 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,298 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,298 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:09,298 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,300 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 11:23:09,300 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:09,300 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@29c2fc40 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@475949e5 11:23:09,304 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7b30f1ae 11:23:09,304 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709032989300, None)) 11:23:09,307 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032989299,0,0)) 11:23:09,307 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@475949e5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7fef313b 11:23:09,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@dea980 11:23:09,308 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 11:23:09,308 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:09,308 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:09,309 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:09,309 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,309 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,309 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,309 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125] 11:23:09,309 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 11:23:09,310 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 11:23:09,311 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:09,314 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,314 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#323464479 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,314 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-1441960247] 11:23:09,314 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:09,314 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,314 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#323464479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,315 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-1441960247] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,316 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:09,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:09,318 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:09,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:23:09,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:23:09,318 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 11:23:09,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 11:23:09,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 11:23:09,320 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,321 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,322 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 11:23:09,322 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 11:23:09,322 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:09,322 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@43a744c6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3b466570 11:23:09,323 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1155549534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,323 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,323 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,323 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:09,323 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,324 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3f79f6c0 11:23:09,324 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709032989322, None)) 11:23:09,324 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032989321,0,0)) 11:23:09,324 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6a9c8e6b 11:23:09,324 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3b466570 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4bcb9159 11:23:09,324 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 11:23:09,324 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:09,324 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,325 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,325 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,325 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053] 11:23:09,325 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 11:23:09,325 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,325 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#113946204] 11:23:09,325 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1021707022 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,325 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1021707022] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,325 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 11:23:09,325 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#113946204] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:09,326 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:09,326 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:09,326 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:23:09,326 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:23:09,326 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 11:23:09,326 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 11:23:09,326 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,326 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 11:23:09,326 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,326 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 11:23:09,374 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#905929556] 11:23:09,375 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 11:23:09,375 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125] 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:23:09,550 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:23:09,551 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 11:23:09,551 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 11:23:09,829 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 11:23:09,829 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:09,829 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 11:23:09,859 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:10,89 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 11:23:10,90 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,90 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 11:23:10,221 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:10,221 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:10,221 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:10,221 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,221 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,221 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:10,221 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,221 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,221 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,221 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,227 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:10,227 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,227 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,227 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:10,227 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,227 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,227 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,300 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:10,320 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 11:23:10,320 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,320 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 11:23:10,320 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,320 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,322 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:10,322 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:10,322 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 11:23:10,322 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:10,323 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:10,323 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 11:23:10,323 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:10,323 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,323 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:10,324 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 11:23:10,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:23:10,325 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:10,328 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:10,328 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,328 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:23:10,328 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:10,328 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:10,328 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:23:10,328 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 11:23:10,329 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 11:23:10,329 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:10,331 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:10,333 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:10,335 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,337 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2076631881] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:10,338 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,339 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,339 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 11:23:10,339 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,339 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:10,339 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7784923 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b7ea3c3 11:23:10,339 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1744fd67 11:23:10,339 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:10,340 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,340 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:10,340 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,340 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:10,340 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,341 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709032990341, None)) 11:23:10,342 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032990340,0,0)) 11:23:10,342 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@8dcff31 11:23:10,342 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b7ea3c3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@54c0f349 11:23:10,342 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,343 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2302f093: creating a new snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#24337228] 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#24337228] 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,343 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,343 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:10,343 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:10,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#24337228] 11:23:10,343 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1155549534] 11:23:10,343 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 11:23:10,343 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} shard 1 11:23:10,343 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 11:23:10,344 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:10,344 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@45f462e0 11:23:10,344 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} scheduling timeout in 2 seconds 11:23:10,344 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:10,344 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 11:23:10,344 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 11:23:10,344 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 11:23:10,344 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#24337228] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:10,344 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1202987458] 11:23:10,344 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1483902104] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:10,344 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1202987458] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:10,344 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1483902104 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,345 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6a9ecda7, attempting to connect 11:23:10,346 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 11:23:10,346 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=6890, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:10,346 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:10,346 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:10,346 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:23:10,346 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:23:10,346 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 11:23:10,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 11:23:10,347 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 11:23:10,347 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,347 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,347 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 11:23:10,348 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@24b2836a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d249e39 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d249e39 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@2f8e3c4a 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@64b388a transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=843678983050} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2302f093 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3f27bb2c 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6dc7485c superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@33f545c2, splicing queue 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6dc7485c queue spliced 0 messages 11:23:10,348 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 537.7 μs 11:23:10,350 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new open snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 11:23:10,350 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-652: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 11:23:10,350 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-652: Message size: 245 bytes, total slices to send: 1 11:23:10,351 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-652: Message does not need to be sliced - sending original message 11:23:10,351 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 11:23:10,351 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=843678983050, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 11:23:10,351 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0} 11:23:10,351 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:10,352 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-655: Closing 11:23:10,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 11:23:10,352 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,352 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:10,352 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:10,355 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:10,355 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:10,355 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:10,356 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:10,357 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:10,371 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,371 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2302f093: creating a new transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:23:10,371 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 11:23:10,371 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:23:10,371 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:10,371 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2302f093 readied transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 11:23:10,371 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@45eabb2e 11:23:10,371 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:10,371 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@64b388a transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], modifications=1, protocol=SIMPLE}, enqueuedTicks=843706680766} 11:23:10,371 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-653: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], modifications=1, protocol=SIMPLE}} 11:23:10,372 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-653: Message size: 407 bytes, total slices to send: 1 11:23:10,372 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-653: Message does not need to be sliced - sending original message 11:23:10,376 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,376 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:23:10,376 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,376 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:23:10,376 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 11:23:10,377 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:23:10,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:23:10,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 11:23:10,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 11:23:10,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:10,991 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 11:23:10,991 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 11:23:10,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:10,991 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 11:23:10,991 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:10,991 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@64b388a superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@640467b3, splicing queue 11:23:10,991 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 11:23:10,991 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@64b388a queue spliced 1 messages 11:23:10,991 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:10,992 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: refreshing backend for shard 1 11:23:10,992 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:10,992 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:10,992 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:10,992 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1155549534] 11:23:10,992 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:10,992 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 11:23:10,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:10,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 11:23:10,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:10,993 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:10,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 11:23:10,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:10,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 11:23:10,996 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 11:23:10,996 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 11:23:10,996 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 11:23:10,996 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:10,996 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 11:23:10,997 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized - notifying 1 OnShardInitialized callbacks 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready - notifying 1 OnShardInitialized callbacks 11:23:10,997 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15add080] 11:23:10,997 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:23:10,997 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15add080] 11:23:10,998 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2974b74c, attempting to connect 11:23:10,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 11:23:10,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=11850, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:10,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:10,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:10,999 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 11:23:10,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:10,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:10,999 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:10,999 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 11:23:10,999 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d249e39 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@604aff16 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@604aff16 11:23:10,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 11:23:10,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 11:23:10,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], snapshotOnly=true} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@53cc892c to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 replayed 1 successful requests 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@604aff16 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], modifications=1, protocol=SIMPLE} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} allocated sequence 0 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@45eabb2e 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1431119c transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], coordinated=false}, enqueuedTicks=843706680766} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2302f093 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@15c40381 11:23:11,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new ready transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@22767dfc superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7c8b01ff, splicing queue 11:23:11,1 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@22767dfc queue spliced 0 messages 11:23:11,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct canCommit 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,2 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.438 ms 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 validated 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct preCommit 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 candidate ready 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - candidate rootPath: / 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - messages: [] 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 prepared 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct commit 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 192 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:11,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 submitted to persistence 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=192} 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 11:23:11,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 0 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data Simple{size=192} 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@aa458d3}, purged=MutableUnsignedLongSet{size=0}} 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committed transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: commit - successfulFromPrevious: [] 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314], coordinated=false}, enqueuedTicks=843706680766, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1667170082] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#571448314]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@60aa4203 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1667170082] 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#571448314]} 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED}: initiating purge 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} allocated sequence 1 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#571448314]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@55c78d9c 11:23:11,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:11,4 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1431119c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#571448314]}, enqueuedTicks=844339620862} 11:23:11,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 11:23:11,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: purging transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 11:23:11,5 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1667170082] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:11,5 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:11,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 11:23:11,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 11:23:11,5 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 102.3 μs 11:23:11,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:11,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:11,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 11:23:11,5 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 11:23:11,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 11:23:11,6 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:11,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 11:23:11,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 11:23:11,6 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#707605053] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#24337228] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#24337228] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:11,6 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 11:23:11,6 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:11,6 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:11,6 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:11,6 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 11:23:11,6 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:11,6 AM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:23:11,7 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2076631881] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2076631881] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:11,7 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:11,7 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2076631881] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#2076631881] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:11,7 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#66347275] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#66347275] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:11,7 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 11:23:11,7 AM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:23:11,7 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:11,7 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 11:23:11,7 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:11,7 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-658: Closing 11:23:11,7 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 11:23:11,8 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 11:23:11,8 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:11,8 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:11,370 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [3663035708762118169] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:12,20 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2949720795770270794] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:12,21 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:12,21 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:12,23 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:12.024] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:12,31 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:12,31 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:12,33 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:12.033] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:12,45 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:12,50 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-937646287801849815] 11:23:12,50 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:12,59 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:12,59 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:12,64 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:12,64 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:12,70 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:12,70 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:12,70 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:12,73 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:12,82 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [566191180830787497] 11:23:12,82 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:12,84 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:12,84 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:12,95 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:12,95 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:12,102 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:12,115 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3512209219564110244] 11:23:12,115 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:12,116 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:12,116 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:12,128 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:12,129 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:12,129 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:12,129 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:12,130 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 11:23:12,142 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1621074620] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,143 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:23:12,145 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:23:12,145 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:23:12,145 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@a54bceb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f044d1c 11:23:12,145 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@647ba13f 11:23:12,145 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 11:23:12,145 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,146 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,146 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:12,146 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,146 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,147 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709032992147, None)) 11:23:12,148 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032992146,0,0)) 11:23:12,148 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f044d1c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@63cd1bdb 11:23:12,148 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 11:23:12,149 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:12,151 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:12,152 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@39317084 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164] 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973] 11:23:12,153 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 11:23:12,153 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,154 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-807094108 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,154 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-807094108] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,154 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:23:12,154 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#608261913] 11:23:12,154 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#608261913] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,155 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:12,155 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:12,157 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:12,157 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 11:23:12,157 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 11:23:12,157 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 11:23:12,157 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 11:23:12,157 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,158 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:23:12,158 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:23:12,158 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:23:12,160 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:12,160 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:12,160 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:12,160 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 11:23:12,161 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 11:23:12,161 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 11:23:12,161 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 11:23:12,164 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1878608143 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,164 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:23:12,164 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:23:12,164 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1878608143] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,165 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 11:23:12,165 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1507862646] 11:23:12,165 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1507862646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,168 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:12,169 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:23:12,173 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#373620923] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,173 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:12,175 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:23:12,175 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:23:12,176 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@bac629d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5b660674 11:23:12,176 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@288c1434 11:23:12,176 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 11:23:12,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,176 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:12,176 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,176 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,176 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709032992176, None)) 11:23:12,176 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032992175,0,0)) 11:23:12,177 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5b660674 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2175fccc 11:23:12,177 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 11:23:12,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@f27541e 11:23:12,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:12,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,178 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,178 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,178 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#2011448359] 11:23:12,178 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,178 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 11:23:12,178 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#2011448359] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,178 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1444992036 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,178 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1444992036] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,178 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:23:12,178 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-696123571] 11:23:12,178 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,178 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,179 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#824066214] 11:23:12,179 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 11:23:12,179 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 11:23:12,179 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-696123571] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,179 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-1869606439 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,179 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-628955467] 11:23:12,185 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:12,187 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:12,190 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#824066214] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,190 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-1869606439] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,190 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-628955467] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:12,191 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:12,191 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:12,191 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:12,191 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:12,191 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 11:23:12,191 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:12,191 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:12,192 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 11:23:12,192 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 11:23:12,192 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 11:23:12,192 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 11:23:12,192 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 11:23:12,192 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,192 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,192 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:23:12,192 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:23:12,192 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 11:23:12,192 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 11:23:12,192 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 11:23:12,192 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 11:23:12,227 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1621074620] 11:23:12,227 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 11:23:12,227 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164] 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:23:12,383 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:12,384 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 11:23:12,384 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:23:12,384 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 11:23:12,444 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:23:12,444 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 11:23:12,673 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:23:12,673 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,673 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:23:12,713 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:23:12,713 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:12,713 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:23:12,752 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:12,791 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:12,943 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:23:12,943 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:12,943 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:23:13,13 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:23:13,13 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,13 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:23:13,85 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:13,85 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,85 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,129 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:13,223 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:23:13,223 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,223 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:23:13,223 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:13,225 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:13,225 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:13,225 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:13,226 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:23:13,226 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 11:23:13,227 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 11:23:13,227 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,227 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 11:23:13,227 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:23:13,227 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:13,232 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:23:13,232 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:13,232 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:13,243 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:23:13,243 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,243 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:23:13,243 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,244 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:23:13,245 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:23:13,245 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:23:13,246 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:13,246 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:13,246 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 11:23:13,246 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 11:23:13,246 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 11:23:13,246 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,246 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 11:23:13,246 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:23:13,246 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:13,246 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:13,247 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 11:23:13,247 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:23:13,247 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:13,287 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1621074620] 11:23:13,288 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 11:23:13,288 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973] 11:23:13,288 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@66ced0c4: creating a new transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:23:13,288 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:23:13,288 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:13,288 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:13,288 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#373620923] 11:23:13,288 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 11:23:13,289 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:23:13,289 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:13,289 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:23:13,289 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:13,289 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:13,289 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#373620923] 11:23:13,289 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 11:23:13,289 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:23:13,289 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:13,289 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@66ced0c4 readied transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:23:13,289 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@26329b1b 11:23:13,289 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:13,289 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@bb76eb4 11:23:13,289 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 11:23:13,289 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:13,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:23:13,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:13,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:23:13,290 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:23:13,290 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 11:23:13,291 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@28442b98, attempting to connect 11:23:13,291 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4acdac1f, attempting to connect 11:23:13,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 11:23:13,293 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 11:23:13,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=15531, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:23:13,293 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=14960, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:23:13,295 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:13,295 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1acf974f instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2f441ae7 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2f441ae7 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@5a79bd1d 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2eb51630 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=846624677165} 11:23:13,295 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-662: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE}} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-662: Message size: 398 bytes, total slices to send: 1 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-662: Message does not need to be sliced - sending original message 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@66ced0c4 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@537db0d7 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@54958c7d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1fef878e, splicing queue 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@54958c7d queue spliced 0 messages 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 851.8 μs 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52837775 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@23a3efe0 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@23a3efe0 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@389d4681 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152a5c86 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=846624562422} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-662: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE}} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-662: Message size: 395 bytes, total slices to send: 1 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-662: Message does not need to be sliced - sending original message 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@66ced0c4 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7be1a814 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3d712ed4 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4cf24b35, splicing queue 11:23:13,296 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3d712ed4 queue spliced 0 messages 11:23:13,297 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 506.6 μs 11:23:13,297 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,297 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,297 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 11:23:13,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 11:23:13,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 11:23:13,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 11:23:13,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 11:23:13,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:13,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 11:23:13,298 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 11:23:13,298 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=846624562422, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 11:23:13,298 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 11:23:13,298 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 11:23:13,298 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=846624677165, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 11:23:13,298 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 11:23:13,298 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:13,299 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@4d27c8f9 11:23:13,299 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:23:13,299 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:23:13,299 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@7f7f6805 11:23:13,299 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:13,299 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152a5c86 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846633982209} 11:23:13,299 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 11:23:13,299 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@4f0d7b90 11:23:13,299 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:23:13,299 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 11:23:13,299 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@54898f6 11:23:13,299 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:13,299 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2eb51630 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846634190534} 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 11:23:13,300 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:13,300 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 11:23:13,301 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 11:23:13,301 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846633982209, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 11:23:13,301 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:13,301 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 11:23:13,301 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846634190534, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 11:23:13,301 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@72608928 11:23:13,301 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@72608928 11:23:13,301 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:13,301 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152a5c86 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846636216830} 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@5695ad0f 11:23:13,301 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 11:23:13,301 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@5695ad0f 11:23:13,301 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:13,301 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2eb51630 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846636369064} 11:23:13,301 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 11:23:13,301 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 11:23:13,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:13,302 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=184} 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:23:13,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:13,303 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 11:23:13,304 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f2ee639}, purged=MutableUnsignedLongSet{size=0}} 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:13,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:23:13,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:13,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:23:13,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:23:13,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 11:23:13,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1925300337] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:23:13,304 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@695d88ac}, purged=MutableUnsignedLongSet{size=0}} 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1925300337] 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 11:23:13,305 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#67079167] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 76.63 μs 11:23:13,305 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:23:13,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1925300337] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#67079167] 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#67079167] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:13,305 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 57.97 μs 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846636216830, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52837775 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@852df82 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@7e856794 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152a5c86 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846641588044} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846636369064, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1acf974f completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 11:23:13,306 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 11:23:13,306 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@66ced0c4: creating a new snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@14c3b08e 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@a73d35b 11:23:13,307 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:13,307 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:13,307 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2eb51630 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846641861610} 11:23:13,307 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@33359276 11:23:13,307 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152a5c86 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=846642025304} 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@470a9a32}, purged=MutableUnsignedLongSet{size=0}} 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-54308455] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@74718756}, purged=MutableUnsignedLongSet{size=0}} 11:23:13,307 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:23:13,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:23:13,308 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,308 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#537315498] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-54308455] 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:23:13,308 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 35.34 μs 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#537315498] 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-54308455] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#537315498] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:13,308 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 70.66 μs 11:23:13,308 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,308 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 11:23:13,308 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 11:23:13,309 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-661: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 11:23:13,309 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message size: 338 bytes, total slices to send: 1 11:23:13,309 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message does not need to be sliced - sending original message 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 11:23:13,309 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 11:23:13,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 11:23:13,309 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,310 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 11:23:13,310 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 11:23:13,310 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=846642025304, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 11:23:13,310 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 11:23:13,310 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:23:13,310 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 11:23:13,310 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@75cebd32 11:23:13,310 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2eb51630 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=846645281019} 11:23:13,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 11:23:13,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:23:13,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 11:23:13,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:13,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:23:13,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 11:23:13,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:13,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 11:23:13,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 11:23:13,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 11:23:13,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:23:13,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:13,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:23:13,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:13,311 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 11:23:13,311 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 11:23:13,311 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 11:23:13,311 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-660: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 11:23:13,311 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-660: Message size: 341 bytes, total slices to send: 1 11:23:13,311 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-660: Message does not need to be sliced - sending original message 11:23:13,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:23:13,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:13,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:23:13,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:23:13,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 11:23:13,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@7a40495a 11:23:13,311 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@33ccf4a9 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:13,312 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#534710555], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=846645281019, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846641861610, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 11:23:13,313 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2f441ae7 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1acf974f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#534710555]}, enqueuedTicks=846641588044, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@23a3efe0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:13,313 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@52837775 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:13,313 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,313 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:13,313 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,313 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 11:23:13,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 11:23:13,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 11:23:13,313 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:13,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5491eeb8 11:23:13,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:13,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:23:13,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5491eeb8 11:23:13,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:23:13,313 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:23:13,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:13,313 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 11:23:13,313 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b528408 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b528408 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:23:13,314 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:13,314 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:23:13,314 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:13,314 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:23:13,314 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:23:13,315 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 11:23:13,315 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 11:23:13,315 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:13,315 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:13,315 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:23:13,315 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152a5c86 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@14ae267c, splicing queue 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152a5c86 queue spliced 0 messages 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 1 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#373620923] 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2eb51630 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@78600cc, splicing queue 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2eb51630 queue spliced 0 messages 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 2 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#483482973], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:23:13,315 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#373620923] 11:23:13,316 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:13,316 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:13,316 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:23:13,316 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:13,733 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:23:13,733 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:13,733 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,733 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-665: Closing 11:23:13,733 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,733 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-666: Closing 11:23:13,734 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:23:13,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:23:13,734 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:23:13,734 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 11:23:13,734 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 11:23:13,734 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 11:23:13,734 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 11:23:13,734 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 11:23:13,735 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1621074620] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1621074620] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:13,736 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:13,736 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:13,740 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:13,740 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:13,740 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:13,740 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:13,743 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:13,754 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 11:23:13,754 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 11:23:13,754 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:23:13,754 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:23:13,755 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 11:23:13,755 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 11:23:13,755 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:23:13,755 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 11:23:13,755 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:13,755 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:14,748 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-937646287801849815] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:14,769 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [566191180830787497] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:14,774 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:14,774 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:14,775 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:14.776] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:14,783 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:14,783 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:23:14.785] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:14,808 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:14,813 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8818616464899742060] 11:23:14,813 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:14,813 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:14,813 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:14,816 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:14,816 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:14,824 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:14,824 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:14,824 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:14,825 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:14,829 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5071521948741537625] 11:23:14,829 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:14,831 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:14,831 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:14,831 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:14,831 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:14,843 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:14,853 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8794145230185545524] 11:23:14,853 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:14,854 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:14,854 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:14,864 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:14,864 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:14,866 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:14,867 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:14,869 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:14,869 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 11:23:14,870 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:14,871 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,872 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 11:23:14,880 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 11:23:14,880 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 11:23:14,880 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 11:23:14,880 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-1-shard-cars-testLargeReadReplySlicing 11:23:14,880 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3f31fb24 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b15c98c 11:23:14,880 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@75e1ed6e 11:23:14,881 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:14,881 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:14,881 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLargeReadReplySlicing, 0, 1709032994881, None)) 11:23:14,881 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032994880,0,0)) 11:23:14,881 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b15c98c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5b541515 11:23:14,881 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} started 11:23:14,881 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:14,882 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:14,884 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 11:23:14,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7db34b67 11:23:14,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:14,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLargeReadReplySlicing 11:23:14,884 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:14,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:14,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-1-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057] 11:23:14,885 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testLargeReadReplySlicing, persistent : true 11:23:14,885 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#550918215 created and ready for shard:member-1-shard-cars-testLargeReadReplySlicing 11:23:14,885 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-1660012429] 11:23:14,885 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLargeReadReplySlicing with journal batch size 1 11:23:14,887 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:14,887 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:14,887 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 11:23:14,887 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 11:23:14,887 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLargeReadReplySlicing is initialized 11:23:14,887 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 11:23:14,887 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 11:23:14,887 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 11:23:14,887 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from null to Follower 11:23:14,893 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,893 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#550918215] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,893 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-1660012429] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,900 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-2115851235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,901 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 11:23:14,905 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 11:23:14,905 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 11:23:14,905 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 11:23:14,905 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-2-shard-cars-testLargeReadReplySlicing 11:23:14,905 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@73a8a30f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57b74807 11:23:14,905 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7537fcb2 11:23:14,906 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLargeReadReplySlicing, 0, 1709032994905, None)) 11:23:14,906 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:14,906 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:14,906 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032994904,0,0)) 11:23:14,906 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57b74807 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1f669a14 11:23:14,906 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} started 11:23:14,907 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5824a352 11:23:14,907 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:14,907 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLargeReadReplySlicing 11:23:14,908 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:14,908 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 11:23:14,908 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-2-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-1667886013] 11:23:14,908 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testLargeReadReplySlicing, persistent : true 11:23:14,908 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-1667886013] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,908 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#1278944853] 11:23:14,909 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#-1762235158 created and ready for shard:member-2-shard-cars-testLargeReadReplySlicing 11:23:14,909 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#-1762235158] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,909 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLargeReadReplySlicing with journal batch size 1 11:23:14,909 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:14,909 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:14,909 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#1278944853] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:14,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 11:23:14,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLargeReadReplySlicing is no longer the leader 11:23:14,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 11:23:14,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLargeReadReplySlicing is initialized 11:23:14,910 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 11:23:14,910 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:14,910 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 11:23:14,910 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-2-shard-cars-testLargeReadReplySlicing from null to Follower 11:23:14,919 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:14,920 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:14,957 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] 11:23:14,957 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: findLocalShard : cars 11:23:14,957 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057] 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Received ElectionTimeout - switching to Candidate 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLargeReadReplySlicing] 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=1, votedFor=member-1-shard-cars-testLargeReadReplySlicing 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 1 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing (Follower): Closing 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 11:23:15,144 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Follower to Candidate 11:23:15,144 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Follower to Candidate 11:23:15,434 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 11:23:15,434 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=2, votedFor=member-1-shard-cars-testLargeReadReplySlicing 11:23:15,434 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 2 11:23:15,504 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:15,705 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 11:23:15,705 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=3, votedFor=member-1-shard-cars-testLargeReadReplySlicing 11:23:15,705 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 3 11:23:15,835 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:15,835 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:15,835 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:15,835 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 11:23:15,835 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:15,835 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:15,835 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 11:23:15,835 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:15,835 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 11:23:15,844 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:15,844 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 11:23:15,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 11:23:15,845 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:15,845 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 11:23:15,845 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 11:23:15,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 11:23:15,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 11:23:15,994 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 4 11:23:15,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLargeReadReplySlicing 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Cluster state: [] 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=null 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 11:23:15,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:15,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:15,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:15,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Election: Leader has following peers: [member-2-shard-cars-testLargeReadReplySlicing] 11:23:15,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:15,997 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:15,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: true, hasLeader : true 11:23:15,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: became leader with frontend state for [] 11:23:15,998 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Candidate to Leader 11:23:15,998 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 11:23:15,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:15,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:15,998 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Candidate to Leader 11:23:15,998 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 11:23:15,998 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing: Closing 11:23:15,998 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: getLeaderAddress leaderId = member-1-shard-cars-testLargeReadReplySlicing peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 11:23:15,998 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done false 11:23:15,999 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 11:23:15,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:15,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:15,999 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 11:23:15,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLargeReadReplySlicing updated: matchIndex: -1, nextIndex: 0 11:23:15,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:16,12 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@36050d7c: creating a new transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 11:23:16,12 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 11:23:16,12 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:16,12 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:16,12 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-2115851235] 11:23:16,12 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=0} shard 1 11:23:16,12 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 11:23:16,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:16,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 11:23:16,13 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@1f0328d8 11:23:16,13 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:16,13 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@2830a95 11:23:16,13 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:16,13 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 11:23:16,14 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 11:23:16,14 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@34190201, attempting to connect 11:23:16,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} is not yet known 11:23:16,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, nanosAgo=11820, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 11:23:16,16 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:16,16 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1084e512 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2ba03229 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2ba03229 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2257/0x00000008016c5568@12d04ea3 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f536cd transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1}, enqueuedTicks=849348039231} 11:23:16,17 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1}} 11:23:16,18 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Message size: 450 bytes, total slices to send: 5 11:23:16,18 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 450, offset: 0, size: 100, index: 1 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341]] 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@42c6f506 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f536cd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=849348039231} 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f536cd cannot transmit request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} - delaying it 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@36050d7c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@71efaf22 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4e68942f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@f458256, splicing queue 11:23:16,25 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4e68942f queue spliced 0 messages 11:23:16,26 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 8.972 ms 11:23:16,31 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,33 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] - creating AssembledMessageState 11:23:16,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 11:23:16,35 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] - expecting more 11:23:16,40 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,40 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 450, offset: 100, size: 100, index: 2 11:23:16,40 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test/user/$a#-1198979341]] 11:23:16,43 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,43 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1085826726, lastSliceHashCode: -1085826726 11:23:16,43 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] - expecting more 11:23:16,44 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,44 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 450, offset: 200, size: 100, index: 3 11:23:16,44 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test/user/$a#-1198979341]] 11:23:16,45 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,45 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: -840689321, lastSliceHashCode: -840689321 11:23:16,45 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] - expecting more 11:23:16,45 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,45 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 450, offset: 300, size: 100, index: 4 11:23:16,46 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=1466515185, replyTo=Actor[akka://cluster-test/user/$a#-1198979341]] 11:23:16,46 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=1466515185, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,46 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: 1466515185, lastSliceHashCode: 1466515185 11:23:16,46 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] - expecting more 11:23:16,46 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,46 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 450, offset: 400, size: 50, index: 5 11:23:16,47 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=50, sliceIndex=5, totalSlices=5, lastSliceHashCode=-818148710, replyTo=Actor[akka://cluster-test/user/$a#-1198979341]] 11:23:16,47 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], data.length=50, sliceIndex=5, totalSlices=5, lastSliceHashCode=-818148710, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,47 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex: 5, lastSliceIndex: 4, assembledSize: 400, sliceHashCode: -818148710, lastSliceHashCode: -818148710 11:23:16,47 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] 11:23:16,48 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] 11:23:16,48 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] was removed from the cache due to EXPLICIT 11:23:16,48 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341], modifications=1}} 11:23:16,48 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testLargeReadReplySlicing: allocating new open transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74], sliceIndex=5, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-669: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=74] was removed from the cache due to EXPLICIT 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f536cd transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=849348039231} 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f536cd transmitted 1 requests 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1}, enqueuedTicks=849348039231, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 11:23:16,49 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0} 11:23:16,66 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 11:23:16,66 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: Message size: 391 bytes, total slices to send: 4 11:23:16,66 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 391, offset: 0, size: 100, index: 1 11:23:16,66 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,71 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,71 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] - creating AssembledMessageState 11:23:16,71 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 11:23:16,71 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] - expecting more 11:23:16,72 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-671: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,72 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,72 AM [cluster-test-shard-dispatcher-22] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 391, offset: 100, size: 100, index: 2 11:23:16,72 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,73 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,73 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1569038920, lastSliceHashCode: -1569038920 11:23:16,73 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] - expecting more 11:23:16,74 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-671: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,74 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,74 AM [cluster-test-shard-dispatcher-22] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 391, offset: 200, size: 100, index: 3 11:23:16,74 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,75 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,75 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: 1439492931, lastSliceHashCode: 1439492931 11:23:16,75 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] - expecting more 11:23:16,77 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-671: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,77 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,77 AM [cluster-test-shard-dispatcher-22] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: getNextSlice: total size: 391, offset: 300, size: 91, index: 4 11:23:16,77 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=91, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,78 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], data.length=91, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] 11:23:16,78 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -31040526, lastSliceHashCode: -31040526 11:23:16,78 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] 11:23:16,78 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] 11:23:16,79 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] was removed from the cache due to EXPLICIT 11:23:16,79 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 11:23:16,79 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 11:23:16,79 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=849348039231, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 11:23:16,79 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1} 11:23:16,79 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 11:23:16,79 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Issuing gracefulStop to shard member-1-shard-cars-testLargeReadReplySlicing 11:23:16,79 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLargeReadReplySlicing - waiting on 1 shards 11:23:16,79 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onShutDown 11:23:16,79 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Initiating leader transfer 11:23:16,80 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 11:23:16,80 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:16,80 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@26d84d10 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testLargeReadReplySlicing: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@26d84d10 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Attempting to transfer leadership 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-671: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-1198979341]] 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] 11:23:16,80 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-668: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=668, messageId=75] was removed from the cache due to EXPLICIT 11:23:16,81 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLargeReadReplySlicing] 11:23:16,81 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done true 11:23:16,81 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 11:23:16,82 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f536cd superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@705f0a9c, splicing queue 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5f536cd queue spliced 0 messages 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: refreshing backend for shard 1 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:16,82 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-2115851235] 11:23:16,82 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:16,82 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLargeReadReplySlicing, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:16,92 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLargeReadReplySlicing, matchIndex: -1, lastIndex: -1, isVoting: true 11:23:16,92 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Follower's log matches - sending ElectionTimeout 11:23:16,92 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:16,92 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Leader transfer complete 11:23:16,494 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: All shards shutdown - sending PoisonPill to self 11:23:16,494 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] ShardManager - shard-manager-testLargeReadReplySlicing: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:16,495 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLargeReadReplySlicing 11:23:16,495 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-671: Closing 11:23:16,495 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 11:23:16,495 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 11:23:16,496 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 11:23:16,496 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 11:23:16,496 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 11:23:16,497 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: shutting down 11:23:16,497 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 11:23:16,497 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:16,497 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:16,497 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:16,497 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1628069977] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:16,500 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:16,500 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:16,504 AM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:16,504 AM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:16,504 AM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:16,504 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:16,505 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:16,516 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Closing 11:23:16,516 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1198979341] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1198979341] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:16,517 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLargeReadReplySlicing 11:23:16,517 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing (Follower): Closing 11:23:16,517 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 11:23:16,517 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 11:23:16,518 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:16,518 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:16,902 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:17,521 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8818616464899742060] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:17,531 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5071521948741537625] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:17,535 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:17,535 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:17,537 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:17.538] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:17,544 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:17,544 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:17,546 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:17.547] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:17,564 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:17,569 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1726239087878786192] 11:23:17,569 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:17,577 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:17,577 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:17,577 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:17,577 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:17,589 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:17,589 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:17,589 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:17,592 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:23:17,596 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8094036992469107360] 11:23:17,596 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:17,599 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:17,599 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:17,599 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:17,599 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:17,613 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:17,618 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [687854830313371833] 11:23:17,619 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:17,623 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:17,623 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:17,623 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:17,623 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:17,628 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:17,629 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:17,631 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 11:23:17,648 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40f009d4 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@70aa9ab9 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:17,648 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2871a3c2 transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-2129855997]}, enqueuedTicks=850982970054} 11:23:17,648 AM [Cleaner-1] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 11:23:17,648 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0} 11:23:17,648 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@48789ed9 aborted transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@377f651c 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013038d0@7710d27 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-98113560], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:17,648 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1409d183 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], modifications=0, protocol=ABORT}, enqueuedTicks=850983276031} 11:23:17,648 AM [Cleaner-1] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-636: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-2095159951], modifications=0, protocol=ABORT}} 11:23:17,648 AM [Cleaner-1] [DEBUG] SlicedMessageState - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-636: Message size: 283 bytes, total slices to send: 1 11:23:17,648 AM [Cleaner-1] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-636: Message does not need to be sliced - sending original message 11:23:17,648 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0} 11:23:17,648 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@367426a aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@6cd77966 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013038d0@8f51575 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#217481153], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:17,648 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60997265 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT}, enqueuedTicks=850983625999} 11:23:17,648 AM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-641: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT}} 11:23:17,648 AM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-641: Message size: 293 bytes, total slices to send: 1 11:23:17,648 AM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-641: Message does not need to be sliced - sending original message 11:23:17,648 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@45fde8e8 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@32c4029 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013038d0@33c5ab93 11:23:17,648 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-613139902], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:23:17,649 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4ad5bfd8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT}, enqueuedTicks=850983854784} 11:23:17,649 AM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-641: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1598814462], modifications=0, protocol=ABORT}} 11:23:17,649 AM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-641: Message size: 294 bytes, total slices to send: 1 11:23:17,649 AM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-641: Message does not need to be sliced - sending original message 11:23:17,649 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0} 11:23:17,649 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@24b2836a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#571448314], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@585b04e6 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 11:23:17,649 AM [Cleaner-1] [DEBUG] RemoteProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} has a successor, which should receive seal through a separate request 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#571448314], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013038d0@72c5bc8b 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1923090125], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:17,649 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@604aff16 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@7b584d30 11:23:17,649 AM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1431119c transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#571448314]}, enqueuedTicks=850984246183} 11:23:17,649 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0} 11:23:17,649 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@23a3efe0 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 1 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#534710555], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@1687c0d3 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#304149164], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:17,649 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1084e512 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@16fb8ad1 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} allocated sequence 2 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1198979341], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013038d0@49854d61 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:17,649 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-146515057], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:17,662 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,667 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:23:17,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 11:23:17,669 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 11:23:17,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:23:17,669 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@c4f0fa1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7cbee1f7 11:23:17,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@13349b62 11:23:17,669 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:17,669 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709032997669, None)) 11:23:17,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032997668,0,0)) 11:23:17,669 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2948b27e 11:23:17,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7cbee1f7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@49c99516 11:23:17,670 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 11:23:17,670 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:17,672 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:17,673 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 11:23:17,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:17,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,674 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275] 11:23:17,675 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 11:23:17,675 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 11:23:17,676 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,676 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1347472380 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,676 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1347472380] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,683 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:17,685 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:17,687 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:17,687 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:17,687 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 11:23:17,687 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 11:23:17,687 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 11:23:17,687 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 11:23:17,687 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 11:23:17,687 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 11:23:17,687 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 11:23:17,692 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#958096675] 11:23:17,692 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#958096675] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,696 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1314606422] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,696 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:23:17,696 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 11:23:17,696 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:23:17,697 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7cb12a97 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3c960a2a 11:23:17,697 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3bcee619 11:23:17,697 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709032997697, None)) 11:23:17,698 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 11:23:17,698 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,698 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709032997696,0,0)) 11:23:17,698 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3c960a2a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@44b55cc 11:23:17,698 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@a17678c 11:23:17,698 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 11:23:17,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:17,699 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,699 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 11:23:17,700 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 11:23:17,700 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1629804364] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,700 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-2041606848 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,700 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-1284800809] 11:23:17,700 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-2041606848] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,701 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-1284800809] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:17,701 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:17,701 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:17,701 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 11:23:17,701 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 11:23:17,701 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 11:23:17,701 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 11:23:17,701 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 11:23:17,701 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 11:23:17,702 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 11:23:17,704 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:17,706 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:17,707 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:17,707 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,707 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,707 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,749 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] 11:23:17,749 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 11:23:17,749 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275] 11:23:17,932 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:17,932 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 11:23:17,932 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 11:23:17,932 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:17,933 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 11:23:17,933 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:17,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 11:23:17,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 11:23:17,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 11:23:17,933 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 11:23:17,933 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 11:23:18,161 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 11:23:18,161 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,162 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 11:23:18,290 AM [cluster-test-shard-dispatcher-15] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:18,412 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 11:23:18,412 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,412 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 11:23:18,584 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:18,584 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:18,584 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:18,584 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,584 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,584 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:18,584 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,584 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,584 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,627 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:18,627 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,627 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,627 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:18,627 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,627 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,632 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:18,672 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 11:23:18,672 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,672 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 11:23:18,672 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,674 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:18,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:18,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:18,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 11:23:18,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:18,676 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:18,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 11:23:18,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 11:23:18,676 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 11:23:18,676 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 11:23:18,676 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:18,676 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:18,676 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 11:23:18,676 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 11:23:18,677 AM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 11:23:18,677 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,677 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 11:23:18,677 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 11:23:18,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:18,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:18,677 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 11:23:18,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 11:23:18,677 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:18,704 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 11:23:18,704 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from IDLE to TX_OPEN 11:23:18,704 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 11:23:18,704 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:18,704 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:18,704 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1314606422] 11:23:18,704 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} shard 1 11:23:18,705 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:18,705 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:23:18,705 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:18,705 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@5ae25682 11:23:18,705 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from TX_OPEN to IDLE 11:23:18,705 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 11:23:18,705 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 11:23:18,705 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@318edbbe 11:23:18,705 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:18,705 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 11:23:18,705 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 11:23:18,706 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5fb05e0d, attempting to connect 11:23:18,708 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} is not yet known 11:23:18,709 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=10530, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:23:18,709 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:18,709 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@639dfa20 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852039896643} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2224/0x00000008016a8238@2e105be3 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY}, enqueuedTicks=852040222061} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY}} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 416 bytes, total slices to send: 1 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2263/0x00000008016c6af0@7ff3a490 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852040360134} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}} 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 296 bytes, total slices to send: 1 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,710 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:18,711 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:23:18,711 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@768675d2 11:23:18,711 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@59bd8fcd superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3b44e86c, splicing queue 11:23:18,711 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@59bd8fcd queue spliced 0 messages 11:23:18,711 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.074 ms 11:23:18,713 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 11:23:18,713 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 11:23:18,713 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:23:18,713 AM [cluster-test-shard-dispatcher-15] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,713 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,713 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,713 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1f01f9d8 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 ready 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 to ready 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 is already in state READY (READY) 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct canCommit 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 validated 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct preCommit 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 candidate ready 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - messages: [] 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 prepared 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct commit 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 11:23:18,714 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 submitted to persistence 11:23:18,715 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 11:23:18,716 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 11:23:18,716 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY}, enqueuedTicks=852040222061, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 11:23:18,716 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0} 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=184} 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:23:18,716 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 1, nextIndex: 2 11:23:18,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:18,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 11:23:18,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 11:23:18,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 11:23:18,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2226/0x00000008016a8a48@7c32981b 11:23:18,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@258dd7a4}, purged=MutableUnsignedLongSet{size=0}} 11:23:18,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:18,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 11:23:18,717 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:18,717 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852039896643, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:18,717 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0} 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 0 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:23:18,718 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2ffdafac}, purged=MutableUnsignedLongSet{size=0}} 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,718 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1857103396] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 11:23:18,720 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1857103396] 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 11:23:18,720 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,720 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 11:23:18,720 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 72.80 μs 11:23:18,721 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1857103396] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852040360134, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1} 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@237d135d 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 11:23:18,722 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from IDLE to TX_OPEN 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6610da05 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:18,722 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852057498568} 11:23:18,722 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:23:18,722 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 11:23:18,723 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@78ad0b55 11:23:18,723 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852057930828} 11:23:18,723 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}} 11:23:18,723 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 456 bytes, total slices to send: 1 11:23:18,723 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,723 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from TX_OPEN to IDLE 11:23:18,723 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 11:23:18,723 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 1 11:23:18,723 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@734d77e9 11:23:18,723 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852058517831} 11:23:18,723 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}} 11:23:18,723 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,723 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 11:23:18,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 11:23:18,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@13d71f8c 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 ready 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 to ready 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 is already in state READY (READY) 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct canCommit 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 validated 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct preCommit 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 candidate ready 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - messages: [] 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 prepared 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct commit 11:23:18,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 submitted to persistence 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 11:23:18,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 11:23:18,728 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 11:23:18,728 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852057930828, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 11:23:18,728 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0} 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 1 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 3 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 11:23:18,728 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,729 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,729 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1324869204] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 11:23:18,729 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 11:23:18,729 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1324869204] 11:23:18,729 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,729 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1324869204] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:18,729 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 57.54 μs 11:23:18,729 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 11:23:18,729 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 11:23:18,729 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@6035c645 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,729 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:23:18,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 11:23:18,730 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,730 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 38.27 μs 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852057498568, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@639dfa20 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852058517831, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} directCommit completed 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@75934457 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:18,731 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852066759581} 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 11:23:18,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 11:23:18,732 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,732 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 45.11 μs 11:23:18,732 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 11:23:18,732 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 11:23:18,732 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@15d1c73d 11:23:18,732 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852067779614} 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}]]] 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,733 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3f62d8d2 11:23:18,734 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 11:23:18,734 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 388 bytes, total slices to send: 1 11:23:18,734 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 11:23:18,734 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 11:23:18,735 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 11:23:18,735 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852067779614, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 11:23:18,735 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0} 11:23:18,735 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}]]] 11:23:18,735 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:23:18,735 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:23:18,735 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 11:23:18,735 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 11:23:18,735 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:23:18,735 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,735 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} 11:23:18,735 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 11:23:18,735 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@61ba543b 11:23:18,735 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852070377404} 11:23:18,735 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}} 11:23:18,735 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,735 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,735 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from IDLE to TX_OPEN 11:23:18,735 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:23:18,735 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} allocated sequence 0 11:23:18,735 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@783508f1 11:23:18,735 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852070715482} 11:23:18,735 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}} 11:23:18,735 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 456 bytes, total slices to send: 1 11:23:18,735 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,736 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from TX_OPEN to IDLE 11:23:18,736 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 11:23:18,736 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} allocated sequence 1 11:23:18,736 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@1dd5a28e 11:23:18,736 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852070924416} 11:23:18,736 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}} 11:23:18,736 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,736 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,736 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 11:23:18,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 4, nextIndex: 5 11:23:18,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 11:23:18,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 11:23:18,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 11:23:18,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 11:23:18,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@7888b47c 11:23:18,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3f62d8d2 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 ready 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 to ready 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 is already in state READY (READY) 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct canCommit 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 validated 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct preCommit 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 candidate ready 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - candidate rootPath: / 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - messages: [] 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: preCommit - successfulFromPrevious: [] 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 prepared 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct commit 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 11:23:18,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 submitted to persistence 11:23:18,740 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 11:23:18,740 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852066759581, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 11:23:18,740 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 11:23:18,740 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} 11:23:18,740 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 11:23:18,740 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852070715482, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 11:23:18,740 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0} 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 4 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,740 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 11:23:18,741 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 3 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2270/0x00000008016ce0f0@6f82c08f 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 11:23:18,741 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: commit - successfulFromPrevious: [] 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:23:18,742 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 11:23:18,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 11:23:18,742 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 78.77 μs 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852070377404, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1} 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN}: initiating purge 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 2 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@1e5d45ad 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852078086531} 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852070924416, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 11:23:18,743 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} directCommit completed 11:23:18,744 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED}: initiating purge 11:23:18,744 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} allocated sequence 2 11:23:18,744 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@7c7b835b 11:23:18,744 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852078869999} 11:23:18,744 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,744 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 11:23:18,744 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,744 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 0 11:23:18,744 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,744 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@5c160282 11:23:18,744 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,744 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852079194567} 11:23:18,744 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7dd6054b 11:23:18,745 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 11:23:18,745 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 11:23:18,745 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 408 bytes, total slices to send: 1 11:23:18,745 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 11:23:18,748 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 11:23:18,748 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852079194567, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 11:23:18,748 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0} 11:23:18,748 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} 11:23:18,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 1 11:23:18,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@3398aa8 11:23:18,748 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852083267480} 11:23:18,748 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}} 11:23:18,748 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,748 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,748 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from IDLE to TX_OPEN 11:23:18,748 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:23:18,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} allocated sequence 0 11:23:18,748 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@55638ed8 11:23:18,748 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852083744091} 11:23:18,748 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=f, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}} 11:23:18,749 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 456 bytes, total slices to send: 1 11:23:18,749 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,749 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from TX_OPEN to IDLE 11:23:18,749 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 11:23:18,749 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} allocated sequence 1 11:23:18,749 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@5e02d1c3 11:23:18,749 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852083976586} 11:23:18,749 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=10, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}} 11:23:18,749 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,749 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,749 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7dd6054b 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 ready 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 to ready 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 is already in state READY (READY) 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct canCommit 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 validated 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct preCommit 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 candidate ready 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - candidate rootPath: / 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - messages: [] 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: preCommit - successfulFromPrevious: [] 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 prepared 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct commit 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 10, leaderSnapShotIndex: 5 11:23:18,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}], SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 submitted to persistence 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,751 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 11:23:18,751 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 11:23:18,751 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852083744091, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 11:23:18,751 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0} 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}], SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 7 11:23:18,751 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 10 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,752 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 11:23:18,752 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,752 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 59.32 μs 11:23:18,753 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 11:23:18,753 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 11:23:18,753 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,753 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 11:23:18,753 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 10, nextIndex: 11 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 6 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@182e40e5 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 8 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@6bdd8941 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 9 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2270/0x00000008016ce0f0@7a9a4844 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 10 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: commit - successfulFromPrevious: [] 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 11:23:18,753 AM [cluster-test-shard-dispatcher-18] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:23:18,754 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,754 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 11:23:18,754 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 11:23:18,754 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 53.29 μs 11:23:18,754 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852078086531, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852078869999, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE}: purge completed 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 11:23:18,759 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 10 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852083267480, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1} 11:23:18,759 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 6, lastIndex(): 10 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN}: initiating purge 11:23:18,759 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 11:23:18,759 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} allocated sequence 2 11:23:18,759 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@1c93291 11:23:18,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852094795555} 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 8 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 11:23:18,760 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 9 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 10 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,760 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 11:23:18,760 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 11:23:18,760 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 73.95 μs 11:23:18,760 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852083976586, sessionId=0, txSequence=16} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 11:23:18,761 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} directCommit completed 11:23:18,761 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED}: initiating purge 11:23:18,761 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} allocated sequence 2 11:23:18,761 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6e976f03 11:23:18,761 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852095895890} 11:23:18,761 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 11:23:18,761 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 0 11:23:18,761 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@2167ebd3 11:23:18,761 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852096174367} 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6d76a260 11:23:18,762 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,762 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 11:23:18,762 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 431 bytes, total slices to send: 1 11:23:18,762 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 11:23:18,762 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 11:23:18,763 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 11:23:18,763 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852096174367, sessionId=0, txSequence=19} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 11:23:18,763 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0} 11:23:18,763 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} 11:23:18,763 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 1 11:23:18,763 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@64ded6db 11:23:18,763 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852098245894} 11:23:18,763 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=14, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}} 11:23:18,763 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,763 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,763 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from IDLE to TX_OPEN 11:23:18,763 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:23:18,763 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} allocated sequence 0 11:23:18,763 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@562fca30 11:23:18,763 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852098545111} 11:23:18,763 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=15, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}} 11:23:18,763 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 456 bytes, total slices to send: 1 11:23:18,763 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,763 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from TX_OPEN to IDLE 11:23:18,763 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 11:23:18,763 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} allocated sequence 1 11:23:18,763 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@72ab2055 11:23:18,763 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852098729725} 11:23:18,763 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=16, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}} 11:23:18,763 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,763 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 13, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6d76a260 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 ready 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 to ready 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 is already in state READY (READY) 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct canCommit 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 validated 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct preCommit 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 candidate ready 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - candidate rootPath: / 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - messages: [] 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: preCommit - successfulFromPrevious: [] 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 prepared 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct commit 11:23:18,764 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 14, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 14, leaderSnapShotIndex: 9 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 submitted to persistence 11:23:18,765 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,765 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 11:23:18,765 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852098545111, sessionId=0, txSequence=21} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 11:23:18,765 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0} 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 11:23:18,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=14, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 14, nextIndex: 15 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 14, lastAppliedIndex: 10 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 11 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@d055ebc 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 12 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@36908ea0 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 13 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2270/0x00000008016ce0f0@27926d52 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 14 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: commit - successfulFromPrevious: [] 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from COMMIT_PENDING to COMMITTED 11:23:18,766 AM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:23:18,766 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,767 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 11:23:18,767 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 14, leaderSnapShotIndex: 13 11:23:18,767 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 58.88 μs 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852094795555, sessionId=0, txSequence=17} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE}: purge completed 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852095895890, sessionId=0, txSequence=18} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE}: purge completed 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852098245894, sessionId=0, txSequence=20} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN}: initiating purge 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} allocated sequence 2 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@1a133f51 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852102495942} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852098729725, sessionId=0, txSequence=22} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} directCommit completed 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED}: initiating purge 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} allocated sequence 2 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@3a521613 11:23:18,767 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852102614185} 11:23:18,767 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 11:23:18,767 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 0 11:23:18,767 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@3fa69ae5 11:23:18,767 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852102791569} 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 14 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 14, context.getLastApplied(): 10, lastIndex(): 14 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 11 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 12 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 13 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 14 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 15, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 11:23:18,768 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 11:23:18,768 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,768 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 46.96 μs 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 16, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,768 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 15 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,769 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}]]] 11:23:18,769 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,769 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,769 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@47fda8ee 11:23:18,769 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 11:23:18,769 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 11:23:18,769 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 451 bytes, total slices to send: 1 11:23:18,769 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 11:23:18,769 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}]]] 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 14, entries to be added from: 0 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=16, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,770 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 11:23:18,770 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852102791569, sessionId=0, txSequence=25} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 11:23:18,770 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0} 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 16, nextIndex: 17 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 16, lastAppliedIndex: 14 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 15 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 11:23:18,770 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 11:23:18,770 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 1 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@5e63975e 11:23:18,770 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@fe45338 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,770 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852105372337} 11:23:18,770 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}} 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 16 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@5cbd7e5e 11:23:18,770 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,770 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 15 for term 4 11:23:18,770 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from IDLE to TX_OPEN 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 17, leaderLastIndex: 16, leaderSnapShotIndex: 15 11:23:18,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=16 11:23:18,770 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:23:18,770 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} allocated sequence 0 11:23:18,770 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@5edc8354 11:23:18,770 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852105745196} 11:23:18,770 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}} 11:23:18,771 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 456 bytes, total slices to send: 1 11:23:18,771 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,771 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 changed state from TX_OPEN to IDLE 11:23:18,771 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6eaece58 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 11:23:18,771 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} allocated sequence 1 11:23:18,771 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@455b4663 11:23:18,771 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852106034112} 11:23:18,771 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1c, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}} 11:23:18,771 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,771 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,771 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,771 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,771 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,771 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 17, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@47fda8ee 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 ready 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 to ready 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 is already in state READY (READY) 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct canCommit 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 validated 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct preCommit 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 candidate ready 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - candidate rootPath: / 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - messages: [] 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: preCommit - successfulFromPrevious: [] 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 prepared 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct commit 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 18, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 17, leaderLastIndex: 18, leaderSnapShotIndex: 15 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 17 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=16, prevLogTerm=4, leaderCommit=16, replicatedToAllIndex=15, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 11:23:18,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 submitted to persistence 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,773 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852102495942, sessionId=0, txSequence=23} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE}: purge completed 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852102614185, sessionId=0, txSequence=24} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE}: purge completed 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY}, enqueuedTicks=852105745196, sessionId=0, txSequence=27} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 11:23:18,773 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0} 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 16 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 16, context.getLastApplied(): 14, lastIndex(): 16 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 15 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 16 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 15 for term 4 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=16, prevLogTerm=4, leaderCommit=16, replicatedToAllIndex=15, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 16, entries to be added from: 0 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 11:23:18,773 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=18, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=16 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 17 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 18, nextIndex: 19 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 18, lastAppliedIndex: 16 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 17 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2270/0x00000008016ce0f0@3cb731d5 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 18 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: commit - successfulFromPrevious: [] 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from COMMIT_PENDING to COMMITTED 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 11:23:18,774 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 19, leaderLastIndex: 18, leaderSnapShotIndex: 17 11:23:18,774 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 62.85 μs 11:23:18,774 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 11:23:18,775 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852105372337, sessionId=0, txSequence=26} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1} 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN}: initiating purge 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} allocated sequence 2 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@72c7e672 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852110889674} 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 18 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 18, context.getLastApplied(): 16, lastIndex(): 18 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 17 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=SIMPLE}, enqueuedTicks=852106034112, sessionId=0, txSequence=28} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} directCommit completed 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED}: initiating purge 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} allocated sequence 2 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@27fd705e 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 18 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852111057668} 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,776 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 11:23:18,776 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 0 11:23:18,776 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@1fc6d189 11:23:18,776 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852111257122} 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 11:23:18,776 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 11:23:18,776 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 52.34 μs 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 19, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 19 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}]]] 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 20, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@62ee826e 11:23:18,777 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 11:23:18,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,777 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 471 bytes, total slices to send: 1 11:23:18,777 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 11:23:18,777 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}]]] 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 18, entries to be added from: 0 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=19, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 19, nextIndex: 20 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 19, lastAppliedIndex: 18 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 19 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@73f72b12 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 18 for term 4 11:23:18,778 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 11:23:18,778 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=852111257122, sessionId=0, txSequence=31} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 20, leaderLastIndex: 20, leaderSnapShotIndex: 18 11:23:18,778 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0} 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 20 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=19, prevLogTerm=4, leaderCommit=19, replicatedToAllIndex=18, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 11:23:18,778 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} 11:23:18,778 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 1 11:23:18,778 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=19 11:23:18,779 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@5da0e191 11:23:18,779 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852113906523} 11:23:18,779 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 11:23:18,779 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=20, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}} 11:23:18,779 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:18,779 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:18,779 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,779 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,779 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,780 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 21, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 11:23:18,780 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852110889674, sessionId=0, txSequence=29} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 11:23:18,780 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE}: purge completed 11:23:18,780 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=19, prevLogTerm=4, leaderCommit=19, replicatedToAllIndex=18, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 19, entries to be added from: 0 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 19 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=20, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 19, context.getLastApplied(): 18, lastIndex(): 20 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 19 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 18 for term 4 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 20, nextIndex: 21 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 20, lastAppliedIndex: 19 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 20 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@1b7bffd9 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 21, leaderLastIndex: 21, leaderSnapShotIndex: 19 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 21 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 11:23:18,780 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=19 11:23:18,781 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 11:23:18,781 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852111057668, sessionId=0, txSequence=30} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 11:23:18,781 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE}: purge completed 11:23:18,781 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 20, entries to be added from: 0 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 20 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=21, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 20, context.getLastApplied(): 19, lastIndex(): 21 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 20 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 11:23:18,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 21, nextIndex: 22 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 21, lastAppliedIndex: 20 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 21 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2270/0x00000008016ce0f0@533e0be6 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 22, leaderLastIndex: 21, leaderSnapShotIndex: 20 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852113906523, sessionId=0, txSequence=32} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1} 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN}: initiating purge 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} allocated sequence 2 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@5afe8d69 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 21 11:23:18,782 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852117700870} 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 21, context.getLastApplied(): 20, lastIndex(): 21 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 21 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 11:23:18,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 11:23:18,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 22, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:18,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,912 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 22 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:18,912 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 21, entries to be added from: 0 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=22, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:18,914 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 22, nextIndex: 23 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 22, lastAppliedIndex: 21 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 22 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@1352fb77 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 11:23:18,915 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 11:23:18,916 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 11:23:18,916 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1587127546]}, enqueuedTicks=852117700870, sessionId=0, txSequence=33} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 11:23:18,916 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE}: purge completed 11:23:18,916 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 22 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 22, context.getLastApplied(): 21, lastIndex(): 22 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 22 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 11:23:18,916 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 11:23:19,280 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] 11:23:19,280 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 11:23:19,280 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275] 11:23:19,280 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275] 11:23:19,281 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 11:23:19,281 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 0 11:23:19,281 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@9f7649f 11:23:19,281 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:19,282 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=852616797420} 11:23:19,283 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 11:23:19,283 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:19,283 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@62ee826e 11:23:19,283 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 11:23:19,283 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message size: 460 bytes, total slices to send: 1 11:23:19,283 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-673: Message does not need to be sliced - sending original message 11:23:19,285 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 11:23:19,285 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1587127546], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=852616797420, sessionId=0, txSequence=34} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 11:23:19,285 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0} 11:23:19,286 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6557d60c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} 11:23:19,286 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 1 11:23:19,286 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@7d833193 11:23:19,286 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:19,286 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}, enqueuedTicks=852620968446} 11:23:19,286 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1587127546], modifications=0, protocol=ABORT}} 11:23:19,286 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 297 bytes, total slices to send: 1 11:23:19,286 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 11:23:19,286 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:23:19,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:19,286 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 11:23:19,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 11:23:19,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6bf9ef21 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6bf9ef21 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:19,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:19,288 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 11:23:19,288 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 11:23:19,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:19,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1587127546], modifications=0, protocol=ABORT}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 11:23:19,288 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:19,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1587127546], modifications=0, protocol=ABORT}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:19,288 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@45f695eb, splicing queue 11:23:19,288 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:19,289 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7f3baa27 queue spliced 1 messages 11:23:19,289 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:19,289 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: refreshing backend for shard 1 11:23:19,289 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:19,289 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:19,289 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:19,289 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1314606422] 11:23:19,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:19,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 22, lastIndex: 22, isVoting: true 11:23:19,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 11:23:19,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:19,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 11:23:19,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 11:23:19,514 AM [cluster-test-shard-dispatcher-15] [WARN] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=22, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 220, lastApplied : 22, commitIndex : 22 11:23:19,602 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:23:19,702 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 11:23:19,703 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-10479275]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:19,703 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:19,703 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-676: Closing 11:23:19,703 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 11:23:19,703 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:23:19,704 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:23:19,704 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 11:23:19,704 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:23:19,704 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 11:23:19,704 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:23:19,704 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:19,704 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:19,704 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1518610874] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:19,705 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:19,705 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:19,709 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:19,709 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:19,709 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:19,709 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:19,711 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:19,728 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 11:23:19,729 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 11:23:19,729 AM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 11:23:19,729 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 11:23:19,729 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 11:23:19,730 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:19,730 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:20,722 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1726239087878786192] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:20,741 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8094036992469107360] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:20,751 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:20,751 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:20,752 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:20.754] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:20,762 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:20,762 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:20,763 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:20.764] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:20,800 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:23:20,805 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-9115362156701499848] 11:23:20,805 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:20,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:20,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:20,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:20,808 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:20,822 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:23:20,824 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:20,824 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:20,824 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:20,828 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5613789918108368323] 11:23:20,828 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:20,835 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:20,835 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:20,840 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:20,840 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:20,848 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:20,860 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2024086569825016515] 11:23:20,860 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:20,861 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:20,861 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:20,872 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:20,874 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:20,875 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 11:23:20,876 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:20,876 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:20,884 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 11:23:20,884 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:23:20,884 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@37f1f13 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1073d16f 11:23:20,887 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1393e727 11:23:20,887 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709033000884, None)) 11:23:20,887 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,887 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033000883,0,0)) 11:23:20,888 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1073d16f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2210537e 11:23:20,888 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 11:23:20,888 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:20,889 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:20,891 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 11:23:20,894 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:20,895 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:20,895 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,895 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:20,895 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,895 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@56fa3aa8 11:23:20,895 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:20,895 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,897 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#551492540] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,898 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:20,900 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,900 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,900 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] 11:23:20,900 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 11:23:20,900 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,900 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1382379792 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,900 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1382379792] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,900 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 11:23:20,900 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-9973890] 11:23:20,900 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-9973890] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,901 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:20,901 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:20,901 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 11:23:20,901 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 11:23:20,903 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 11:23:20,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:20,903 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:23:20,903 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6ac75c60 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d476380 11:23:20,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,903 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7845ea4c 11:23:20,903 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709033000903, None)) 11:23:20,904 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033000902,0,0)) 11:23:20,904 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d476380 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@198f18aa 11:23:20,904 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 11:23:20,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@69f0cfdd 11:23:20,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:20,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,905 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:23:20,905 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:23:20,905 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 11:23:20,905 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 11:23:20,905 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 11:23:20,906 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 11:23:20,907 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 11:23:20,907 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#419294516 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,907 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#679382685] 11:23:20,907 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-1898984211] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,907 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#419294516] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,907 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#679382685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:20,907 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:20,907 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:20,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 11:23:20,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 11:23:20,907 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:23:20,907 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:23:20,908 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 11:23:20,908 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 11:23:20,908 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 11:23:20,912 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:20,914 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:20,914 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,914 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,914 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:20,914 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:20,916 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:20,919 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:20,954 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] 11:23:20,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 11:23:20,954 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:23:21,178 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 11:23:21,178 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 11:23:21,450 AM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:21,457 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 11:23:21,457 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,457 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 11:23:21,738 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 11:23:21,738 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,738 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 11:23:21,819 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:21,819 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:21,819 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:21,819 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,819 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,819 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:21,819 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,819 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,819 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,820 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,824 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:21,824 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,824 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,824 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:21,824 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,824 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,824 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:21,881 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:22,8 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 11:23:22,8 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,8 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 11:23:22,8 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,8 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,9 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:22,9 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:22,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:22,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:23:22,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:22,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:22,10 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:22,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 11:23:22,11 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 11:23:22,11 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 11:23:22,11 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:23:22,11 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:22,11 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:22,11 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 11:23:22,11 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:23:22,11 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,11 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,11 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,11 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:23:22,12 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:22,12 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:22,12 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:23:22,12 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 11:23:22,12 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 11:23:22,12 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:22,61 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3235796c: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:23:22,61 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 11:23:22,61 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:22,61 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:22,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] 11:23:22,61 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} shard 1 11:23:22,61 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:23:22,61 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:22,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:22,61 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3235796c readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 11:23:22,61 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@28670d3c 11:23:22,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@13f04e4b] 11:23:22,61 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:22,61 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@13f04e4b] 11:23:22,62 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@255b577d, attempting to connect 11:23:22,62 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} is not yet known 11:23:22,62 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=9590, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:23:22,62 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:22,62 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:22,62 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,62 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5c4e9a2f instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@317fe610 11:23:22,62 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:22,62 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@317fe610 11:23:22,62 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,62 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], modifications=1, protocol=SIMPLE} 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} allocated sequence 0 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@28670d3c 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7b301dd6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false}, enqueuedTicks=855396765528} 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3235796c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct canCommit 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@35313bc6 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 validated 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b6cb842 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4fe7b63b, splicing queue 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct preCommit 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b6cb842 queue spliced 0 messages 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 candidate ready 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - messages: [] 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 prepared 11:23:22,63 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.020 ms 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct commit 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:22,63 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:22,64 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:22,64 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:22,64 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:22,64 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,64 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 11:23:22,64 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 submitted to persistence 11:23:22,65 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:22,65 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:22,65 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@566802a7}, purged=MutableUnsignedLongSet{size=0}} 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1935165626] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false}, enqueuedTicks=855396765528, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@34c2dcbc 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 11:23:22,66 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED}: initiating purge 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} allocated sequence 1 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1935165626] 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@8913ee9 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7b301dd6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]}, enqueuedTicks=855401324473} 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 60.35 μs 11:23:22,66 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1935165626] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 11:23:22,66 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 11:23:22,67 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7d90cbfb}, purged=MutableUnsignedLongSet{size=0}} 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,67 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-229402804] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 11:23:22,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 11:23:22,68 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 11:23:22,68 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-229402804] 11:23:22,68 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:23:22,68 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 50.63 μs 11:23:22,68 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-229402804] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,68 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,68 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 11:23:22,68 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:22,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 11:23:22,71 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 11:23:22,76 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:22,77 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1452643846] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: replication of member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@6d6b5cd7 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: finished purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 11:23:22,79 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 11:23:22,80 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]}, enqueuedTicks=855401324473, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 11:23:22,80 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE}: purge completed 11:23:22,80 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@317fe610 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} 11:23:22,80 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:22,80 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5c4e9a2f purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,81 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 11:23:22,82 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 11:23:22,82 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:22,82 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7d13a09a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@791d3513 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,82 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@11869db8 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:22,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,83 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709033002083, None)) 11:23:22,83 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033002082,0,0)) 11:23:22,83 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1bc0130d 11:23:22,83 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@791d3513 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@50e6c18e 11:23:22,83 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,84 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1452643846] 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-57255419] 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-57255419] 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,84 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-57255419] 11:23:22,84 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#551492540] 11:23:22,84 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3235796c: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:23:22,84 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 11:23:22,84 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:23:22,84 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 11:23:22,85 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] 11:23:22,85 AM [pool-35-thread-1] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3235796c readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 11:23:22,85 AM [pool-35-thread-1] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 11:23:22,85 AM [pool-35-thread-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@6b975e99 11:23:22,85 AM [pool-35-thread-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,85 AM [pool-35-thread-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7b301dd6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false}, enqueuedTicks=855420198857} 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,85 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-57255419] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct canCommit 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,85 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-320981900 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 validated 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct preCommit 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,85 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-320981900] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 candidate ready 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - candidate rootPath: / 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - messages: [] 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 prepared 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct commit 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 72 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 submitted to persistence 11:23:22,86 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:22,87 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 11:23:22,87 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:23:22,87 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,87 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,88 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,88 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 11:23:22,88 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 11:23:22,88 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:22,88 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 11:23:22,88 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,88 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:23:22,88 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 11:23:22,88 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:22,88 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:22,88 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 11:23:22,88 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:23:22,88 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-1465142528] 11:23:22,88 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-1465142528] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,186 AM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] 11:23:22,187 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106] 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-681: Closing 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,189 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 11:23:22,189 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 11:23:22,190 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:22,190 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7b301dd6 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@765ecf59, splicing queue 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7b301dd6 queue spliced 1 messages 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:22,190 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] 11:23:22,190 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:22,191 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 11:23:22,427 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:22,427 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 11:23:22,427 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:23:22,427 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,427 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 11:23:22,427 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,428 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,428 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 11:23:22,428 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:23:22,428 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 11:23:22,428 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 11:23:22,428 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 11:23:22,428 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 11:23:22,428 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,429 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 11:23:22,430 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 11:23:22,430 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 11:23:22,430 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:22,430 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:22,430 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 11:23:22,430 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:22,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:22,430 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 11:23:22,431 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 11:23:22,431 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:23:22,431 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:23:22,432 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 11:23:22,432 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:22,432 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 11:23:22,432 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:22,432 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 11:23:22,433 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 11:23:22,434 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:22,434 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:22,434 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 11:23:22,434 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:22,434 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 11:23:22,434 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@35900cc3}, purged=MutableUnsignedLongSet{size=0}} 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,435 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:22,435 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-268710826] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 11:23:22,435 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-268710826] 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 11:23:22,436 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 11:23:22,436 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 67.12 μs 11:23:22,436 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-268710826] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,436 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 11:23:22,436 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:22,436 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, size=72} 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: commit - successfulFromPrevious: [] 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:23:22,436 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:23:22,436 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} not found in inflight queue, checking pending queue 11:23:22,436 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]] - lastApplied: 3, lastIndex: 3 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509], coordinated=false}, enqueuedTicks=855420198857, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 32.08 μs 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} directCommit completed 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED}: initiating purge 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 1 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-684: Closing 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@76412911 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,437 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 11:23:22,437 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is ready - notifying 1 OnShardInitialized callbacks 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@13f04e4b] 11:23:22,437 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:22,437 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 11:23:22,437 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 11:23:22,438 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 11:23:22,438 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 11:23:22,438 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@13f04e4b] 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] is present: existingEntryTerm: 4 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] is present: existingEntryTerm: 6 11:23:22,438 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 2 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:22,438 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:23:22,438 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 11:23:22,438 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 17.52 μs 11:23:22,438 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 11:23:22,438 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 11:23:22,438 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:22,438 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@713d9ed3, attempting to connect 11:23:22,438 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 11:23:22,438 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 11:23:22,439 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 16.81 μs 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@317fe610 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6ba555a 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} previous state DONE 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6ba555a 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN}: initiating purge 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 0 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@7eadb84f 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@574c3a2d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509]}, enqueuedTicks=855771986180} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3235796c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@388a51b 11:23:22,439 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 not tracked in MutableUnsignedLongSet{span=[0..0], size=1}, but not present in active transactions 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@76557357 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@65a9d212, splicing queue 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@76557357 queue spliced 0 messages 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 580.7 μs 11:23:22,439 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 11:23:22,440 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-528020509]}, enqueuedTicks=855771986180, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 11:23:22,440 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE}: purge completed 11:23:22,440 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6ba555a purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} 11:23:22,440 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 11:23:22,440 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@317fe610 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 11:23:22,491 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:23:22,491 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,491 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 11:23:22,491 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 11:23:22,491 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,492 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:23:22,492 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 11:23:22,492 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 11:23:22,492 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:22,492 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,492 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:23:22,493 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 11:23:22,493 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 11:23:22,493 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72a36afe 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72a36afe 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:23:22,493 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1452643846] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1452643846] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:23:22,493 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-725957563] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-725957563] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 11:23:22,493 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@574c3a2d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@530201d8, splicing queue 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@574c3a2d queue spliced 0 messages 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:22,494 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] 11:23:22,494 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 11:23:22,495 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 11:23:22,495 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 11:23:22,495 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:22,495 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:22,495 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:22,495 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 11:23:22,496 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 11:23:22,496 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 11:23:22,496 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:23:22,496 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:23:22,496 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 11:23:22,839 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:23:22,908 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 11:23:22,908 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1148802106]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:22,908 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,908 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-685: Closing 11:23:22,908 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 11:23:22,909 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:22,909 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,909 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,909 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 11:23:22,909 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,909 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 11:23:22,909 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,909 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,909 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#100114500] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:22,910 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:22,910 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:22,913 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:22,913 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:22,913 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:22,914 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:22,915 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:22,920 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 11:23:22,920 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 11:23:22,920 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 11:23:22,920 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 11:23:22,921 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 11:23:22,925 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:22,925 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:23,930 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-9115362156701499848] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:23,940 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5613789918108368323] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:23,942 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:23,942 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:23,946 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:23.947] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:23,961 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:23,961 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:23,963 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:23.965] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:23,993 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:24,0 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6851083254942204025] 11:23:24,0 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:24,3 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:24,3 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:24,3 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:24,3 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:24,15 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:24,15 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:24,15 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:24,17 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:24,27 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7411093476262136067] 11:23:24,27 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:24,33 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:24,33 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:24,33 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:24,33 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:24,62 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:24,62 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:24,65 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:24,71 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3368261856313924625] 11:23:24,72 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:24,73 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:24,73 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:24,73 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:24,73 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:24,84 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:24,85 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:24,86 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 11:23:24,90 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 11:23:24,90 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-1201788043] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,97 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 11:23:24,97 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 11:23:24,97 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:23:24,97 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,97 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@26c2b645 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e0b4f24 11:23:24,97 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5b09d281 11:23:24,97 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709033004097, None)) 11:23:24,97 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:24,97 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,97 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033004096,0,0)) 11:23:24,97 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3a76d5b 11:23:24,97 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e0b4f24 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@74dd0f84 11:23:24,97 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 11:23:24,98 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:24,98 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,98 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:24,98 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,98 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,98 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669] 11:23:24,98 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 11:23:24,98 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,98 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#230289275 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,98 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#230289275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,98 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1817755304] 11:23:24,98 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1817755304] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,99 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:24,100 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 11:23:24,101 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 11:23:24,102 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:24,102 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:24,102 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 11:23:24,102 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:23:24,102 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 11:23:24,102 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 11:23:24,102 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 11:23:24,102 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 11:23:24,102 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 11:23:24,103 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-456595968] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,105 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 11:23:24,108 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 11:23:24,108 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 11:23:24,108 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:23:24,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 11:23:24,108 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@ec86594 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@26e33ba0 11:23:24,108 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@63e1b118 11:23:24,108 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:24,108 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,108 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709033004108, None)) 11:23:24,109 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033004107,0,0)) 11:23:24,109 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@26e33ba0 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6dc94b87 11:23:24,109 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 11:23:24,109 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@14624201 11:23:24,109 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:24,109 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 11:23:24,110 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,110 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:23:24,110 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-2-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#910663077] 11:23:24,110 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 11:23:24,110 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:24,110 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 11:23:24,111 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#910663077] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,111 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1166387038] 11:23:24,111 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-1750472394] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,111 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1166387038] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:24,111 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-1750472394 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 11:23:24,112 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:24,113 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:24,113 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:24,113 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:23:24,113 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 11:23:24,113 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 11:23:24,113 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 11:23:24,113 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 11:23:24,113 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,113 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 11:23:24,114 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 11:23:24,159 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-1201788043] 11:23:24,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 11:23:24,160 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669] 11:23:24,405 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:24,405 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:23:24,405 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 11:23:24,405 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,405 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 11:23:24,406 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:24,406 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 11:23:24,406 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:23:24,406 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 11:23:24,406 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 11:23:24,406 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 11:23:24,644 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:24,695 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 11:23:24,695 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,695 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 11:23:24,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 11:23:24,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:23:24,945 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 11:23:25,16 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:25,16 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:25,16 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:25,16 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 11:23:25,16 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:25,16 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:25,16 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,16 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:25,17 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,22 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:25,22 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 11:23:25,22 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,22 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:25,22 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,22 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,93 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:25,175 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 11:23:25,175 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:23:25,175 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 11:23:25,175 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,176 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:25,177 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 11:23:25,177 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:25,177 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:25,177 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 11:23:25,177 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:25,177 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 11:23:25,177 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 11:23:25,178 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 11:23:25,178 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 11:23:25,179 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 11:23:25,179 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 11:23:25,179 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 11:23:25,179 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:25,179 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:25,179 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 11:23:25,179 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 11:23:25,179 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:23:25,179 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:25,215 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 11:23:25,215 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 changed state from IDLE to TX_OPEN 11:23:25,216 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 11:23:25,216 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:25,216 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:25,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-456595968] 11:23:25,216 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:25,216 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} shard 1 11:23:25,216 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 11:23:25,216 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:25,216 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:23:25,216 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:25,216 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@4ec52351 11:23:25,216 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 changed state from TX_OPEN to IDLE 11:23:25,216 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:23:25,216 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 11:23:25,216 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 11:23:25,216 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@10db0805 11:23:25,217 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 11:23:25,217 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@755d6cde, attempting to connect 11:23:25,220 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} is not yet known 11:23:25,221 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=13180, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:23:25,221 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@23af6687 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state SEALED 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} previous state OPEN 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858551234932} 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2224/0x00000008016a8238@dbc239c 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY}, enqueuedTicks=858551522419} 11:23:25,222 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY}} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message size: 389 bytes, total slices to send: 1 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@773cc3fd 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=858551871277} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2059200a 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@71afff25 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@74bf2344, splicing queue 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@71afff25 queue spliced 0 messages 11:23:25,223 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.478 ms 11:23:25,224 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,224 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,224 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:23:25,224 AM [cluster-test-shard-dispatcher-23] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,224 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,224 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,224 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6c950e9a 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 ready 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 to ready 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}] 11:23:25,225 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 11:23:25,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,225 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message size: 331 bytes, total slices to send: 1 11:23:25,225 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 11:23:25,225 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 11:23:25,225 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY}, enqueuedTicks=858551522419, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 11:23:25,225 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0} 11:23:25,226 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 11:23:25,226 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=858551871277, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 11:23:25,226 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0} 11:23:25,226 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 changed state from IDLE to TX_OPEN 11:23:25,226 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:23:25,226 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 11:23:25,226 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@79ac944d 11:23:25,226 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=858561397226} 11:23:25,227 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,227 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,227 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}] 11:23:25,227 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 11:23:25,227 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message size: 331 bytes, total slices to send: 1 11:23:25,227 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 11:23:25,228 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 11:23:25,228 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=858561397226, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 11:23:25,228 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0} 11:23:25,228 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 11:23:25,228 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 2 11:23:25,228 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@12560a1f 11:23:25,228 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2}, enqueuedTicks=858563565046} 11:23:25,228 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2}} 11:23:25,228 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message size: 459 bytes, total slices to send: 1 11:23:25,229 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 11:23:25,229 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303698@51590356 11:23:25,229 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=858564055187} 11:23:25,231 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 11:23:25,231 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2}, enqueuedTicks=858563565046, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 11:23:25,231 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1} 11:23:25,231 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 11:23:25,231 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=858564055187, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 11:23:25,231 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true} 11:23:25,231 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 3 11:23:25,231 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@43f82f0e 11:23:25,231 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=858566210227} 11:23:25,233 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 11:23:25,233 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message size: 369 bytes, total slices to send: 1 11:23:25,233 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 11:23:25,234 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 11:23:25,234 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=858566210227, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 11:23:25,234 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3} 11:23:25,234 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 4 11:23:25,234 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 11:23:25,234 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@70fbe100 11:23:25,234 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE}, enqueuedTicks=858569730017} 11:23:25,234 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE}} 11:23:25,235 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message size: 285 bytes, total slices to send: 1 11:23:25,235 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 11:23:25,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 is already in state READY (READY) 11:23:25,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct canCommit 11:23:25,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 validated 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct preCommit 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 candidate ready 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - messages: [] 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 prepared 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct commit 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 submitted to persistence 11:23:25,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=174} 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 11:23:25,237 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 1, nextIndex: 2 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2226/0x00000008016a8a48@43f8810a 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@230e2905}, purged=MutableUnsignedLongSet{size=0}} 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 11:23:25,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:25,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 11:23:25,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-330951212] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-330951212] 11:23:25,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:23:25,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:23:25,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-330951212] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:25,238 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,238 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 81.00 μs 11:23:25,238 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858551234932, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE}, enqueuedTicks=858569730017, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} directCommit completed 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED}: initiating purge 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@658d3505 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:25,238 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@12846825 11:23:25,238 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858573697079} 11:23:25,238 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2, protocol=READY}, enqueuedTicks=858573770290} 11:23:25,238 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2, protocol=READY}} 11:23:25,239 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message size: 473 bytes, total slices to send: 1 11:23:25,239 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 11:23:25,239 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 changed state from TX_OPEN to IDLE 11:23:25,239 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:23:25,239 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} allocated sequence 5 11:23:25,239 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@4cd8cb91 11:23:25,239 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE}, enqueuedTicks=858574100248} 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 1 11:23:25,239 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE}} 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 11:23:25,239 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message size: 286 bytes, total slices to send: 1 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 11:23:25,239 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1a41329f}, purged=MutableUnsignedLongSet{size=0}} 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#690655793] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:23:25,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 11:23:25,239 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,239 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 11:23:25,239 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 11:23:25,239 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#690655793] 11:23:25,239 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:25,239 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:23:25,239 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,240 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 39.36 μs 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}]]] 11:23:25,240 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#690655793] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 ready 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 to ready 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 is already in state READY (READY) 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct canCommit 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 validated 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct preCommit 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 candidate ready 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - candidate rootPath: / 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - messages: [] 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: preCommit - successfulFromPrevious: [] 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 prepared 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct commit 11:23:25,240 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 230 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 submitted to persistence 11:23:25,241 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 11:23:25,241 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2, protocol=READY}, enqueuedTicks=858573770290, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 11:23:25,241 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4} 11:23:25,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}]]] 11:23:25,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 11:23:25,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:23:25,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 11:23:25,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 11:23:25,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 11:23:25,241 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:25,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 11:23:25,243 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 2, nextIndex: 3 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@75bb7e02 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,243 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:23:25,244 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:23:25,244 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,244 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 11:23:25,244 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 11:23:25,244 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292]} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:25,244 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858573697079, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:25,244 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 11:23:25,244 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=230} 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 2 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 3 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 11:23:25,245 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: commit - successfulFromPrevious: [] 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 11:23:25,245 AM [cluster-test-shard-dispatcher-23] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:23:25,246 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 11:23:25,246 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 97.21 μs 11:23:25,246 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=SIMPLE}, enqueuedTicks=858574100248, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} directCommit completed 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED}: initiating purge 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 6 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 11:23:25,246 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#502075292]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@5f426c5a 11:23:25,246 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:25,246 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@23af6687 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 11:23:25,246 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 1 11:23:25,246 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858581549119} 11:23:25,246 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,246 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@2ce77f53 11:23:25,246 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 11:23:25,246 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} allocated sequence 1 11:23:25,246 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x00000008013038d0@268a1200 11:23:25,246 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT}, enqueuedTicks=858581747514} 11:23:25,246 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=c, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT}} 11:23:25,246 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message size: 286 bytes, total slices to send: 1 11:23:25,246 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 11:23:25,247 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 11:23:25,247 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,247 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 11:23:25,247 AM [main] [WARN] ClientBackedTransactionChain - Aborted unclosed transactions [member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0] java.lang.Throwable: at at org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain.close(ClientBackedTransactionChain.java:76) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testTransactionChainWithSingleShard(DistributedDataStoreRemotingIntegrationTest.java:581) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 11:23:25,247 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 invoking destroy 11:23:25,247 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 61.77 μs 11:23:25,247 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858582245555} 11:23:25,247 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 11:23:25,247 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 11:23:25,247 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1 changed state from IDLE to CLOSED 11:23:25,247 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2356d95: creating a new snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 11:23:25,247 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0} shard 1 11:23:25,247 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 11:23:25,247 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:25,247 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@1ec9e26f 11:23:25,247 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=858582445790} 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: aborting transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 11:23:25,248 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 11:23:25,248 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message size: 368 bytes, total slices to send: 1 11:23:25,248 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-687: Message does not need to be sliced - sending original message 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 11:23:25,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,249 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 11:23:25,249 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#502075292], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=858582445790, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 11:23:25,249 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0} 11:23:25,249 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:23:25,249 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 11:23:25,249 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 11:23:25,249 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b6e4de2 11:23:25,249 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:25,249 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b6e4de2 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,249 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 3 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4a583d1, splicing queue 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7ad17311 queue spliced 3 messages 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 4 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: refreshing backend for shard 1 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:25,250 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-456595968] 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:25,250 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:25,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 11:23:25,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 11:23:25,251 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 11:23:25,251 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 6, nextIndex: 7 11:23:25,251 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 3 11:23:25,251 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 4 11:23:25,251 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 11:23:25,251 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 11:23:25,251 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@7d4e6e07 11:23:25,251 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 5 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2270/0x00000008016ce0f0@54e62d11 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 6 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2241/0x00000008016b6668@5418549 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 4 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 6, lastIndex: 6, isVoting: true 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 11:23:25,252 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} not found in inflight queue, checking pending queue 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#502075292]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858581549119, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 6 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT}, enqueuedTicks=858581747514, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 3, lastIndex(): 6 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 4 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR}: initiating purge 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 5 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 2 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@747f6145 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 6 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 11:23:25,252 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 4 11:23:25,252 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED}: initiating purge 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#502075292]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@5ceccf10 11:23:25,253 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292]} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292]}, enqueuedTicks=858582245555, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1} 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3bb96cd1: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@756d1434 11:23:25,253 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:25,664 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 11:23:25,664 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:25,665 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 11:23:25,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-690: Closing 11:23:25,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 11:23:25,665 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 11:23:25,665 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:23:25,665 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 11:23:25,665 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:23:25,666 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 11:23:25,666 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:23:25,666 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:25,666 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:25,678 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:25,678 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:25,678 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:25,678 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:25,679 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:25,687 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 11:23:25,688 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 11:23:25,688 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 11:23:25,688 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 11:23:25,688 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 11:23:25,689 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:25,689 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:26,683 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6851083254942204025] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:26,702 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7411093476262136067] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:26,707 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:26,707 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:26,709 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:26.710] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:26,715 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:26,715 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:26,717 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:26.718] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:26,728 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:26,734 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8543062077045170040] 11:23:26,734 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:26,735 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:26,735 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:26,738 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:26,738 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:26,747 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:26,747 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:26,747 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:26,751 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:26,755 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1295611222925328548] 11:23:26,755 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:26,757 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:26,757 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:26,763 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:26,763 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:26,770 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:26,777 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2303761547849908678] 11:23:26,777 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:26,780 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:26,780 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:26,782 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:26,782 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:26,789 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:26,790 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:26,791 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:26,791 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:26,796 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:26,796 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:26,798 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:26,799 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:23:26,799 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 11:23:26,811 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:26,811 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:27,830 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:27.831] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:27,841 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:27,841 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:27,844 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:27.845] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:27,856 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:27,861 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7720406826773316229] 11:23:27,862 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:27,863 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:27,863 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:27,863 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:27,863 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:27,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:27,877 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:27,877 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:27,877 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:27,883 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1003506159613326581] 11:23:27,883 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:27,884 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:27,884 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:27,893 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:27,893 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:27,908 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:27,914 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2409130022572132647] 11:23:27,914 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:27,916 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:27,916 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:27,919 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:27,919 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:27,926 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:27,927 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:27,928 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 11:23:27,936 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:23:27,937 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,948 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 11:23:27,948 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:23:27,949 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@57527b8b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@50bdaebf 11:23:27,949 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@518fc380 11:23:27,949 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 11:23:27,949 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,949 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709033007949, None)) 11:23:27,949 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:27,949 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,950 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033007948,0,0)) 11:23:27,950 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@50bdaebf to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@653733cd 11:23:27,950 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 11:23:27,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@75ee4328 11:23:27,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:27,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,951 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:27,951 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614] 11:23:27,951 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 11:23:27,951 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,951 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#460734633 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,951 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#460734633] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,952 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-400985611] 11:23:27,952 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-400985611] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,953 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:27,955 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 11:23:27,962 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:23:27,967 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1986575093] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,973 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:27,981 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 11:23:27,982 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:27,982 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:27,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 11:23:27,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 11:23:27,982 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:23:27,982 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 11:23:27,982 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 11:23:27,982 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 11:23:27,982 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 11:23:27,985 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 11:23:27,985 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:23:27,985 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@261c83ca to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4ab7f5f2 11:23:27,985 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@46573fe6 11:23:27,985 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:27,985 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 11:23:27,985 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,986 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709033007985, None)) 11:23:27,986 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033007984,0,0)) 11:23:27,986 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@622d99d4 11:23:27,986 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4ab7f5f2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@242cfc87 11:23:27,986 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 11:23:27,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:27,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,988 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:27,990 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:27,995 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:27,995 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,995 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,995 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 11:23:27,995 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#2145655768] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,996 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1907085397 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,996 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-5561419] 11:23:27,996 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1907085397] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,996 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-5561419] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:27,996 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 11:23:27,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:27,997 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:27,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 11:23:27,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 11:23:27,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:23:27,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 11:23:27,997 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 11:23:27,997 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:27,997 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 11:23:27,998 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 11:23:28,36 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] 11:23:28,37 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 11:23:28,37 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614] 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 11:23:28,253 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 11:23:28,253 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 11:23:28,573 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:23:28,573 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,573 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 11:23:28,596 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:28,833 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:23:28,833 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,833 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 11:23:28,883 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:28,884 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:28,884 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:28,884 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,884 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,884 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:28,884 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,884 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,884 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,921 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:28,921 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,921 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:28,921 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:28,927 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:29,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 11:23:29,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,133 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 11:23:29,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,135 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:29,136 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:29,136 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:29,136 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 11:23:29,136 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 11:23:29,137 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 11:23:29,137 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 11:23:29,137 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 11:23:29,138 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,138 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 11:23:29,138 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 11:23:29,138 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:29,138 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:29,138 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 11:23:29,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 11:23:29,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:29,193 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1986575093] 11:23:29,193 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 11:23:29,193 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#2145655768] 11:23:29,193 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,193 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1986575093] 11:23:29,193 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 11:23:29,193 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#2145655768] 11:23:29,204 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,204 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,204 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 11:23:29,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 11:23:29,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:23:29,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-275-0 of 3 operations, client version 13 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-275-0 ready 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 validated 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-275-0: success 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 candidate ready 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: canCommit - candidate rootPath: / 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: canCommit - messages: [] 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: preCommit - successfulFromPrevious: [] 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 prepared 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 198 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-275-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 submitted to persistence 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:23:29,209 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 11:23:29,209 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 11:23:29,209 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=3304476} is already disabled 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}]]] 11:23:29,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:23:29,210 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}]]] 11:23:29,210 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:23:29,210 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:29,210 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=198} 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 11:23:29,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=198} 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: commit - successfulFromPrevious: [] 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from COMMIT_PENDING to COMMITTED 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-765#709746373] 11:23:29,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#949797203] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 11:23:29,212 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:23:29,212 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:29,212 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:23:29,212 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#949797203] 11:23:29,212 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:23:29,212 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 86.63 μs 11:23:29,212 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#949797203] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:29,213 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@32886b05: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 11:23:29,213 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 11:23:29,213 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:29,213 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:29,213 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] 11:23:29,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 11:23:29,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:23:29,213 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:29,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 11:23:29,213 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 11:23:29,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 11:23:29,213 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@fd81715] 11:23:29,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=198} 11:23:29,213 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 11:23:29,213 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:29,213 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@fd81715] 11:23:29,213 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#180582504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@5452ffa1 11:23:29,213 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:29,213 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3ae05afc, attempting to connect 11:23:29,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-275-0 11:23:29,213 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} is not yet known 11:23:29,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, nanosAgo=5760, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:23:29,214 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7d30173a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3af30fa0 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:29,214 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-2128951838] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3af30fa0 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-2128951838] 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#180582504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@32886b05 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@19232e27 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@74622be3 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@564ea4c0, splicing queue 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@74622be3 queue spliced 0 messages 11:23:29,214 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 80.57 μs 11:23:29,214 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 346.8 μs 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 11:23:29,214 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-2128951838] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:29,216 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,216 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,217 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 11:23:29,217 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,218 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-276-0 of 1 operations, client version 13 11:23:29,218 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,218 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-276-0 ready 11:23:29,218 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 canCommit 11:23:29,218 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614)] resolved 11:23:29,218 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: successfully resolved all cohort actors 11:23:29,218 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 finishCanCommit 11:23:29,218 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-276-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614)] 11:23:29,218 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614)] 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 validated 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-276-0: success 11:23:29,219 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: received CanCommitTransactionReply [canCommit=true, version=13] 11:23:29,219 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit returning result: true 11:23:29,219 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 commit 11:23:29,219 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 finish commit 11:23:29,219 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-276-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614)] 11:23:29,219 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614)] 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 candidate ready 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit - candidate rootPath: / 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit - messages: [] 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: preCommit - successfulFromPrevious: [] 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 prepared 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 209 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-276-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}]]] 11:23:29,220 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 submitted to persistence 11:23:29,220 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}]]] 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=209} 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:29,225 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 11:23:29,225 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=209} 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: commit - successfulFromPrevious: [] 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from COMMIT_PENDING to COMMITTED 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23738892614$a] 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:23:29,226 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:23:29,226 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: commit succeeded 11:23:29,226 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@32886b05: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 11:23:29,226 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:23:29,226 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 77.10 μs 11:23:29,227 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:23:29,227 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,227 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 11:23:29,227 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 11:23:29,227 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 11:23:29,227 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:23:29,227 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@21ec31fb 11:23:29,227 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:29,227 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@21ec31fb 11:23:29,227 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 11:23:29,227 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@12e59326 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@6629ca95, splicing queue 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@12e59326 queue spliced 0 messages 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: refreshing backend for shard 1 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:29,227 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] 11:23:29,227 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 11:23:29,228 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 11:23:29,228 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 11:23:29,228 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 11:23:29,228 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=209} 11:23:29,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:29,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 11:23:29,228 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-276-0 11:23:29,228 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 11:23:29,228 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 11:23:29,228 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 11:23:29,228 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 72.84 μs 11:23:29,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 11:23:29,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 11:23:29,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:23:29,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 11:23:29,229 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 11:23:29,229 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 11:23:29,229 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:29,229 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:29,643 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 11:23:29,643 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:29,643 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,643 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-695: Closing 11:23:29,644 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:23:29,644 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:23:29,644 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:23:29,644 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 11:23:29,644 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:29,644 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 11:23:29,644 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:23:29,644 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:23:29,644 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#180582504] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#180582504] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:29,644 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:29,644 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#180582504] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#180582504] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:29,645 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1618446669] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:29,646 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:29,646 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:29,651 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:29,651 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:29,651 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:29,651 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:29,654 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:29,664 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 11:23:29,665 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 11:23:29,665 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 11:23:29,665 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 11:23:29,665 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 11:23:29,665 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:29,665 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:30,657 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7720406826773316229] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:30,678 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1003506159613326581] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:30,679 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:30,679 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:30,681 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:30.682] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:30,688 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:30,688 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:30,689 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:30.690] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:30,704 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:30,709 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2903284860049921809] 11:23:30,709 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:30,710 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:30,710 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:30,711 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:30,711 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:30,721 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:30,721 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:30,721 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:30,722 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:30,730 AM [cluster-test-akka.actor.default-dispatcher-9] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6885905640866923169] 11:23:30,730 AM [cluster-test-akka.actor.default-dispatcher-9] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:30,731 AM [cluster-test-akka.actor.default-dispatcher-9] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:30,731 AM [cluster-test-akka.actor.default-dispatcher-9] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:30,736 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:30,736 AM [cluster-test-akka.actor.default-dispatcher-9] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:30,747 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:30,757 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4300181059874506162] 11:23:30,759 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:30,760 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:30,765 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:30,765 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:30,765 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:30,768 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:30,771 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:30,772 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 11:23:30,774 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:30,780 AM [cluster-test-akka.actor.default-dispatcher-9] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:30,782 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-620374893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,786 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:23:30,788 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 11:23:30,789 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:23:30,789 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 11:23:30,789 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@414c3a7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d82ff46 11:23:30,789 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,789 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@69bd662e 11:23:30,789 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709033010789, None)) 11:23:30,789 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:30,789 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,789 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033010788,0,0)) 11:23:30,789 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:30,789 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d82ff46 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4db7828e 11:23:30,789 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 11:23:30,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@37ae1d57 11:23:30,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:30,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,790 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:30,790 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,790 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820] 11:23:30,790 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 11:23:30,790 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,791 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1664540583 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,791 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1664540583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,791 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-763780680] 11:23:30,791 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-763780680] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,791 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 11:23:30,791 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:30,793 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 11:23:30,794 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:30,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:30,794 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:30,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:23:30,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 11:23:30,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 11:23:30,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 11:23:30,794 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 11:23:30,796 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:23:30,797 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 11:23:30,797 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 11:23:30,797 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1825099112] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,798 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 11:23:30,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,798 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:30,798 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,799 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 11:23:30,799 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:23:30,799 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7d9bfe95 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2cab269a 11:23:30,799 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2894b7d7 11:23:30,799 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709033010799, None)) 11:23:30,799 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033010798,0,0)) 11:23:30,799 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2cab269a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@247d9743 11:23:30,799 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4fb1503c 11:23:30,799 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 11:23:30,800 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:30,800 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,800 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,800 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,800 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1653346954] 11:23:30,800 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 11:23:30,800 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1653346954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,801 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-859919460 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,801 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#2010181253] 11:23:30,801 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-859919460] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,801 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 11:23:30,801 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#2010181253] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:30,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:30,801 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:30,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:23:30,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 11:23:30,801 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 11:23:30,801 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 11:23:30,801 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:30,802 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 11:23:30,802 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 11:23:30,802 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 11:23:30,850 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-620374893] 11:23:30,850 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 11:23:30,850 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820] 11:23:31,52 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:31,52 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 11:23:31,52 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 11:23:31,52 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,53 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 11:23:31,53 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:31,53 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 11:23:31,53 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 11:23:31,53 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 11:23:31,53 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 11:23:31,53 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 11:23:31,362 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 11:23:31,362 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,362 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 11:23:31,372 AM [cluster-test-shard-dispatcher-20] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:31,652 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 11:23:31,652 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,652 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 11:23:31,723 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:31,723 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:31,723 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:31,723 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,723 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,723 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:31,723 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,723 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,723 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,727 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:31,728 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,728 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,728 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:31,728 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,728 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,774 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:31,932 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 11:23:31,932 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,932 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 11:23:31,932 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,933 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:31,933 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 11:23:31,933 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:31,933 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:31,933 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 11:23:31,934 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:31,934 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,934 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 11:23:31,935 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 11:23:31,935 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 11:23:31,936 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 11:23:31,936 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,936 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 11:23:31,936 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:31,936 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:31,936 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 11:23:31,936 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 11:23:31,937 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 11:23:31,937 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:31,957 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ac67009 11:23:31,957 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 1675960650, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@1d241613} 11:23:31,958 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ac67009 changed state from IDLE to TX_OPEN 11:23:31,958 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 11:23:31,958 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:31,958 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:31,958 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1825099112] 11:23:31,958 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:31,958 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} shard 1 11:23:31,958 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:23:31,958 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 11:23:31,958 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 11:23:31,958 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:31,958 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 11:23:31,958 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2222/0x00000008016a3580@441dcab4 11:23:31,958 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ac67009 changed state from TX_OPEN to IDLE 11:23:31,958 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ac67009 readied transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 11:23:31,958 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 11:23:31,958 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 11:23:31,958 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@1f026899 11:23:31,958 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@76dcdb51, attempting to connect 11:23:31,960 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} is not yet known 11:23:31,960 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=37381, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:23:31,961 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2a15d2aa instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1742715a 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50911a0c transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080]}, enqueuedTicks=865293418687} 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1742715a 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2224/0x00000008016a8238@88a7428 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50911a0c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY}, enqueuedTicks=865293636302} 11:23:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-698: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY}} 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-698: Message size: 486 bytes, total slices to send: 1 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-698: Message does not need to be sliced - sending original message 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2263/0x00000008016c6af0@45e878d 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50911a0c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE}, enqueuedTicks=865293807366} 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-698: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE}} 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-698: Message size: 294 bytes, total slices to send: 1 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-698: Message does not need to be sliced - sending original message 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ac67009 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4a4d6a6 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@13aeb0ae superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@60383108, splicing queue 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@13aeb0ae queue spliced 0 messages 11:23:31,962 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.386 ms 11:23:31,963 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 11:23:31,963 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 11:23:31,964 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 11:23:31,964 AM [cluster-test-shard-dispatcher-29] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 11:23:31,964 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 11:23:31,964 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 11:23:31,964 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@41c5c074 11:23:31,964 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:31,964 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 11:23:31,965 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=0, protocol=SIMPLE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:31,965 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=1, protocol=READY} failed to process}} 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY}, enqueuedTicks=865293636302, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=1, protocol=READY} failed to process}} 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=1, protocol=READY} failed to process} 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#116354080], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE}, enqueuedTicks=865293807366, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#116354080], modifications=0, protocol=SIMPLE} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0} 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:31,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:31,968 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@57819b8c failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$directCommit$4(AbstractProxyTransaction.java:517) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$16(RemoteProxyTransaction.java:509) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 11:23:31,979 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6e01996b aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#502075292], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@708db8ac 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1327732669], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 11:23:31,979 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0} 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 11:23:31,979 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7d30173a aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#180582504], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@58246f36 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#116354080]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@3044b637 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 11:23:31,979 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3af30fa0 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#180582504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@1243cbe1 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#116354080]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@420101d2 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 11:23:31,979 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50911a0c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#116354080]}, enqueuedTicks=865314121803} 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:31,979 AM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3af30fa0 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#180582504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1215/0x00000008012fc048@71ce728d 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 11:23:31,979 AM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#738892614], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:31,980 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: purging transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 11:23:31,980 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 11:23:31,980 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 11:23:31,980 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:31,980 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 11:23:31,989 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@57819b8c is no longer running 11:23:31,989 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:23:31,989 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,989 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 11:23:31,989 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 11:23:31,990 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4a256a19 11:23:31,990 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:31,990 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4a256a19 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:31,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}]]] 11:23:31,990 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 11:23:31,991 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 11:23:31,991 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}]]] 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 2 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50911a0c superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1af04cd4, splicing queue 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 2 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50911a0c queue spliced 2 messages 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: refreshing backend for shard 1 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:31,991 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1825099112] 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:31,991 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 11:23:31,992 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 11:23:31,992 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 11:23:31,992 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 1, nextIndex: 2 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2226/0x00000008016a8a48@606f5b9c 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4f065b9}, purged=MutableUnsignedLongSet{size=0}} 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 1 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 11:23:31,992 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@8b79cd9 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: finished purging transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Purged transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 11:23:31,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#116354080]}, enqueuedTicks=865293418687, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} not found in inflight queue, checking pending queue 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#116354080]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#116354080]}, enqueuedTicks=865314121803, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1742715a purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 11:23:31,994 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 1 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 11:23:31,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2a15d2aa purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 11:23:31,994 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3b4d75e0}, purged=MutableUnsignedLongSet{size=0}} 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 1 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Purged transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 11:23:31,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 11:23:32,402 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 11:23:32,402 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#-336676820]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:23:32,402 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:32,402 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-700: Closing 11:23:32,402 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 11:23:32,402 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:23:32,403 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:23:32,403 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 11:23:32,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 11:23:32,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:23:32,403 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:23:32,403 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1011493964] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1011493964] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:32,403 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-620374893] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-620374893] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:32,403 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1011493964] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1011493964] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:32,403 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-620374893] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-620374893] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:32,404 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:32,404 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:32,409 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:32,409 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:32,409 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:32,409 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:32,426 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 11:23:32,426 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 11:23:32,426 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 11:23:32,426 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 11:23:32,426 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 11:23:32,427 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:32,427 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:33,424 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2903284860049921809] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:33,444 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:33,444 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:33,445 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6885905640866923169] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:33,445 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:33.447] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:33,452 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:33,452 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:33,454 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:33.455] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:33,471 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:33,478 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2816966649490633280] 11:23:33,478 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:33,479 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:33,479 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:33,479 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:33,479 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:33,489 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:33,489 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:33,489 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:33,493 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:33,498 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4349968887199256542] 11:23:33,498 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:33,500 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:33,500 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:33,500 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:33,500 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:33,515 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:33,521 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8638672754709217872] 11:23:33,522 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:33,523 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:33,523 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:33,528 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:33,528 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:33,531 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:33,535 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:33,536 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:33,537 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 11:23:33,539 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:33,541 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:23:33,544 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1963858038] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,544 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 11:23:33,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,545 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:33,545 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,546 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 11:23:33,546 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:23:33,546 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@18c08c0b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@64de3885 11:23:33,546 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@60e1a0c6 11:23:33,546 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709033013546, None)) 11:23:33,546 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033013545,0,0)) 11:23:33,546 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@64de3885 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6d205811 11:23:33,546 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5870de6c 11:23:33,546 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 11:23:33,547 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:33,548 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:33,550 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:33,550 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 11:23:33,550 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:33,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:33,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,551 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657] 11:23:33,551 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 11:23:33,551 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,552 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1861648037 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,552 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1861648037] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,552 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-863357880] 11:23:33,552 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 11:23:33,552 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-863357880] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,553 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:33,553 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:33,553 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 11:23:33,553 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 11:23:33,553 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 11:23:33,553 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 11:23:33,553 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 11:23:33,553 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 11:23:33,553 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 11:23:33,554 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#575975022] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,554 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 11:23:33,554 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:23:33,554 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@269ab7dc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@67a6c6db 11:23:33,554 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@63994ef7 11:23:33,554 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:23:33,554 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709033013554, None)) 11:23:33,554 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033013553,0,0)) 11:23:33,555 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@67a6c6db to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@75a59027 11:23:33,555 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 11:23:33,559 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 11:23:33,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,559 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:33,559 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4cab7a02 11:23:33,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:33,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,560 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,560 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,560 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#1788521672] 11:23:33,560 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 11:23:33,560 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#1788521672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,560 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1921111547 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,560 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1921111547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,560 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-1739338428] 11:23:33,560 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-1739338428] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:33,560 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 11:23:33,560 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:33,560 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:33,561 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 11:23:33,561 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 11:23:33,561 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 11:23:33,561 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,561 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 11:23:33,561 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 11:23:33,562 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 11:23:33,564 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 11:23:33,605 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-1963858038] 11:23:33,605 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 11:23:33,605 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657] 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 11:23:33,857 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 11:23:33,857 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 11:23:33,858 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 11:23:34,126 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 11:23:34,126 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,127 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 11:23:34,377 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 11:23:34,377 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,377 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 11:23:34,497 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:34,497 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:34,498 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:34,498 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,498 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,498 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:34,498 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,498 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,498 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,505 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:34,505 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,505 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,505 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:34,505 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,505 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,542 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:34,627 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 11:23:34,627 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,627 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 11:23:34,627 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,628 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:34,628 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 11:23:34,629 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:34,629 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:34,629 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 11:23:34,629 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:34,629 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,629 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:34,630 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:34,630 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:34,630 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 11:23:34,630 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:34,630 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:34,630 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 11:23:34,630 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 11:23:34,631 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 11:23:34,631 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,631 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 11:23:34,631 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 11:23:34,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:34,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:34,631 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 11:23:34,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 11:23:34,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:34,633 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 11:23:34,633 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 11:23:34,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:34,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:34,633 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 11:23:34,633 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 11:23:34,660 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19f4bf88: creating a new snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 11:23:34,661 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 11:23:34,661 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:34,661 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:34,661 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#575975022] 11:23:34,661 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=0} shard 1 11:23:34,661 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 11:23:34,661 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:34,661 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@1d6f11b2 11:23:34,661 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} scheduling timeout in 2 seconds 11:23:34,661 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:34,662 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 11:23:34,662 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 11:23:34,662 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3c7c9cae, attempting to connect 11:23:34,664 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} is not yet known 11:23:34,664 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=14011, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:23:34,665 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@494f6e2c instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6bf83dd6 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6bf83dd6 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1248/0x0000000801308d10@13a53fd3 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1995161919 nanoseconds 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1df18fdc transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=867996535957} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19f4bf88 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@37cd1739 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@32c0ed0f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@8a11723, splicing queue 11:23:34,666 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@32c0ed0f queue spliced 0 messages 11:23:34,667 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 760.3 μs 11:23:34,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: allocating new open snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 11:23:34,670 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-702: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 11:23:34,670 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-702: Message size: 235 bytes, total slices to send: 1 11:23:34,670 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-702: Message does not need to be sliced - sending original message 11:23:34,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 11:23:34,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=867996535957, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 11:23:34,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0} 11:23:34,671 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:23:34,671 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,671 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-705: Closing 11:23:34,671 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 11:23:34,672 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:23:34,672 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:34,672 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:34,676 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:34,676 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:34,676 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:34,676 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:34,678 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:34,689 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 11:23:34,689 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19f4bf88: creating a new transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 11:23:34,689 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 11:23:34,689 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:34,689 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@19f4bf88 readied transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 11:23:34,689 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@2ed48048 11:23:34,690 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:34,690 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:34,690 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1df18fdc transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], modifications=1, protocol=SIMPLE}, enqueuedTicks=868024845037} 11:23:34,690 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-703: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], modifications=1, protocol=SIMPLE}} 11:23:34,690 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:34,690 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-703: Message size: 399 bytes, total slices to send: 1 11:23:34,690 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-703: Message does not need to be sliced - sending original message 11:23:35,691 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2816966649490633280] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:35,691 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2816966649490633280] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:36,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 11:23:36,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} is empty 11:23:36,671 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 11:23:36,682 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:36,682 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 7670135 nanoseconds 11:23:36,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:36,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1649242091], modifications=1, protocol=SIMPLE}, enqueuedTicks=868024845037, sessionId=0, txSequence=1} 11:23:36,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 11:23:36,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:36,712 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} directCommit completed 11:23:36,712 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED}: initiating purge 11:23:36,712 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} allocated sequence 1 11:23:36,712 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1649242091]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@19bff684 11:23:36,712 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1013594657], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999933939 nanoseconds 11:23:36,712 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1df18fdc transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1649242091]}, enqueuedTicks=870047042059} 11:23:36,712 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:23:36,713 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:23:36,713 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:23:36,713 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 11:23:36,713 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:23:36,714 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:23:36,714 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 11:23:36,714 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 11:23:36,714 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 11:23:36,714 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 11:23:36,714 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 11:23:36,715 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:36,715 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:37,732 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4349968887199256542] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:37,732 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:37,732 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:37,733 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:37.735] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:37,743 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:37,743 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:37,744 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:37.744] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:37,759 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 11:23:37,764 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4197593050156667490] 11:23:37,764 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:37,767 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:37,767 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:37,772 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:37,772 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:37,779 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:37,779 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:37,779 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:37,780 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:37,784 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3495342063657260769] 11:23:37,784 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:37,786 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:37,786 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:37,792 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:37,792 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:37,801 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:37,807 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1635764675027670864] 11:23:37,807 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:37,809 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:37,809 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:37,813 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:37,813 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:37,821 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:37,822 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:37,823 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 11:23:37,834 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#127187749] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,837 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:37,838 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:37,840 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 11:23:37,840 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:23:37,840 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@297ede17 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2409551e 11:23:37,840 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@610bcffb 11:23:37,840 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:37,840 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,840 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:37,841 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,841 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709033017841, None)) 11:23:37,841 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033017840,0,0)) 11:23:37,841 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2409551e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3eef4485 11:23:37,841 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 11:23:37,842 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:37,843 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:37,844 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 11:23:37,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5f88c01c 11:23:37,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:37,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,847 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,847 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,847 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107] 11:23:37,847 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 11:23:37,848 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,848 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 11:23:37,848 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-605692621] 11:23:37,848 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1746487154 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,848 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1746487154] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,848 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-605692621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,849 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:37,849 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:37,849 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:37,849 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:37,849 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:37,849 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 11:23:37,849 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 11:23:37,850 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 11:23:37,850 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 11:23:37,850 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 11:23:37,850 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:37,853 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:37,856 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,857 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:37,858 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 11:23:37,858 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:23:37,858 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@475ace8f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@24bfb00c 11:23:37,858 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@65f29985 11:23:37,858 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:37,858 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,858 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:37,858 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,859 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709033017858, None)) 11:23:37,859 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033017857,0,0)) 11:23:37,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@17fd7c11 11:23:37,859 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@24bfb00c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@17ec2c44 11:23:37,859 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 11:23:37,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:37,859 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,860 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,860 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,860 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933] 11:23:37,860 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 11:23:37,860 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,860 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#190318896 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,860 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 11:23:37,860 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#1473967839] 11:23:37,860 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#190318896] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,860 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#1473967839] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:37,860 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:37,860 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:37,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:37,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:37,861 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:37,861 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 11:23:37,861 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 11:23:37,861 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 11:23:37,861 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 11:23:37,861 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 11:23:37,909 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#127187749] 11:23:37,910 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 11:23:37,910 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107] 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:38,116 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:23:38,116 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:23:38,416 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:38,416 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,416 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 11:23:38,419 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:38,646 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:38,646 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,646 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 11:23:38,786 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:38,786 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:38,787 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:38,787 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,787 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,787 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:38,787 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,787 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,787 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,936 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:38,936 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,936 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 11:23:38,936 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:38,938 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:38,938 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:38,938 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:38,938 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:38,938 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:38,938 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 11:23:38,939 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 11:23:38,939 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 11:23:38,939 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:23:38,939 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:38,939 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:38,939 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 11:23:38,939 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:38,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:38,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,939 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:23:38,939 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:38,939 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:38,939 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:38,939 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 11:23:38,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 11:23:38,945 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:38,966 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:23:38,966 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 11:23:38,966 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:38,966 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:38,966 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:38,967 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:38,967 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} shard 1 11:23:38,967 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:23:38,967 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 11:23:38,967 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:38,967 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 11:23:38,967 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 11:23:38,967 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@5ed89323 11:23:38,967 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6844fbd4, attempting to connect 11:23:38,967 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} scheduling timeout in 2 seconds 11:23:38,969 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} is not yet known 11:23:38,969 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=10991, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:23:38,970 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@30b1f8 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@67c356b8 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@67c356b8 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2263/0x00000008016c6af0@36f05f9b 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@72a9211b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE}, enqueuedTicks=872302247157} 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-708: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE}} 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-708: Message size: 413 bytes, total slices to send: 1 11:23:38,970 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-708: Message does not need to be sliced - sending original message 11:23:38,971 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:38,971 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 11:23:38,971 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4f221749 11:23:38,971 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@79976135 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7f036ad1, splicing queue 11:23:38,971 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@79976135 queue spliced 0 messages 11:23:38,971 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 862.6 μs 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 ready 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 to ready 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct canCommit 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 validated 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct preCommit 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 candidate ready 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - messages: [] 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 prepared 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct commit 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 182 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 11:23:38,972 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 submitted to persistence 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=182} 11:23:38,973 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@645a2fda}, purged=MutableUnsignedLongSet{size=0}} 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:23:38,974 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:38,975 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2009114086] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 11:23:38,975 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:38,975 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE}, enqueuedTicks=872302247157, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=2, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0} 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@d3ba025 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED}: initiating purge 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 1 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@527c6694 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:38,975 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@72a9211b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]}, enqueuedTicks=872310714472} 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7134a15d}, purged=MutableUnsignedLongSet{size=0}} 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#152477399] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 11:23:38,976 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 11:23:38,976 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#152477399] 11:23:38,976 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 11:23:38,976 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 53.14 μs 11:23:38,976 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#152477399] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:38,977 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:38,977 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:38,977 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:38,977 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:38,978 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2009114086] 11:23:38,978 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 11:23:38,978 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 67.25 μs 11:23:38,978 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2009114086] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:38,978 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:38,980 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 11:23:38,981 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,981 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 11:23:38,981 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,981 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:38,981 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 11:23:38,981 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:38,981 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:38,981 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:38,981 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-710: Closing 11:23:38,981 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 11:23:38,981 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:38,982 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:38,982 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:23:38.985] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1082)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:23:38.985] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1082)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:23:38.985] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1082)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:23:38.985] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1082)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:38,998 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,3 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:39,47 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:39,48 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 11:23:39,48 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933] 11:23:39,48 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,98 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,148 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,188 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 11:23:39,188 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 11:23:39,188 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 11:23:39,188 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:39,189 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:39,189 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:23:39,189 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:39,189 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@72a9211b superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1551d40e, splicing queue 11:23:39,189 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 11:23:39,189 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@72a9211b queue spliced 1 messages 11:23:39,190 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:39,190 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 11:23:39,190 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:39,190 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:39,190 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:39,190 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:39,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:39,190 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 11:23:39,204 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:39,211 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6366852359189618374] 11:23:39,211 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:39,213 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:39,213 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:39,216 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:39,216 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:39,221 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:39,222 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:39,224 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 11:23:39,226 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#304471951] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:39,227 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:39,228 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:23:39,228 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:39,229 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,229 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 11:23:39,229 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@766202d1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@38686584 11:23:39,229 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6c38270c 11:23:39,229 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709033019229, None)) 11:23:39,229 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033019228,0,0)) 11:23:39,229 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@38686584 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@35b322dc 11:23:39,229 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 11:23:39,229 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6a530a35 11:23:39,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:39,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,230 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 11:23:39,230 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 11:23:39,231 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:39,231 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#558790253] 11:23:39,233 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1066064089 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,233 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#558790253] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:39,233 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1066064089] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:23:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:23:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:23:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,237 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 182 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@224c601c}, purged=MutableUnsignedLongSet{size=0}} 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 74 11:23:39,238 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:39,239 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 443.9 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 11:23:39,239 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 11:23:39,239 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 11:23:39,239 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:39,239 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:39,239 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 11:23:39,239 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 11:23:39,239 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 11:23:39,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:39,280 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 11:23:39,280 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933] 11:23:39,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:39,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,368 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 11:23:39,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 11:23:39,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 11:23:39,409 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:39,409 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:39,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:39,409 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:39,518 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:39,518 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,518 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 11:23:39,519 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:39,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,659 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 11:23:39,659 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:39,761 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:39,762 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:23:39,762 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:23:39,848 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:39,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:39,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,879 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 11:23:39,879 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:39,997 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4197593050156667490] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:39,999 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-4197593050156667490] but current is [6366852359189618374]. Reason: ActorSystem terminated 11:23:40,20 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#-1531450817]] intended for system uid [-4197593050156667490], self uid is [6366852359189618374] 11:23:40,31 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] LocalActorRefProvider$$anon$2 - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [3495342063657260769]. This system with UID [6366852359189618374] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 5 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=false] 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:40,32 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:23:40,32 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 11:23:40,32 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#598917061]] intended for system uid [-4197593050156667490], self uid is [6366852359189618374] 11:23:40,32 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 11:23:40,32 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 11:23:40,33 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:40,33 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=false], my term is 9 11:23:40,33 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=false], current voteCount: 1 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 6 - updating term 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=false] 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 7 - updating term 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 11:23:40,34 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=false] 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 8 - updating term 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,35 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=false] 11:23:40,35 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=7, voteGranted=false], my term is 9 11:23:40,35 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=7, voteGranted=false], current voteCount: 1 11:23:40,35 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=8, voteGranted=false], my term is 9 11:23:40,35 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=8, voteGranted=false], current voteCount: 1 11:23:40,35 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=9, voteGranted=false], my term is 9 11:23:40,35 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=9, voteGranted=false], current voteCount: 1 11:23:40,79 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:40,79 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,79 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 11:23:40,79 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:40,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:40,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:40,80 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 9 - updating term 11:23:40,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 11:23:40,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=false] 11:23:40,80 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=10, voteGranted=false], my term is 10 11:23:40,80 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=10, voteGranted=false], current voteCount: 1 11:23:40,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:40,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,209 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 11 11:23:40,209 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:40,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:40,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:40,210 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 11 in "RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 10 - updating term 11:23:40,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=null 11:23:40,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 11, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=11, voteGranted=false] 11:23:40,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=11, voteGranted=false], my term is 11 11:23:40,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=11, voteGranted=false], current voteCount: 1 11:23:40,233 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 11:23:40,348 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:40,348 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,348 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 12 11:23:40,348 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,349 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:40,349 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:40,349 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:40,349 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 12 in "RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 11 - updating term 11:23:40,349 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=null 11:23:40,349 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 12, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,349 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=12, voteGranted=false] 11:23:40,351 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=12, voteGranted=false], my term is 12 11:23:40,351 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=12, voteGranted=false], current voteCount: 1 11:23:40,429 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:40,429 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:40,429 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:40,429 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:40,429 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:40,429 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 11:23:40,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:40,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,559 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 13 11:23:40,559 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:40,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:40,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:40,560 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 13 in "RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 12 - updating term 11:23:40,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=null 11:23:40,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 13, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=13, voteGranted=false] 11:23:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=13, voteGranted=false], my term is 13 11:23:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=13, voteGranted=false], current voteCount: 1 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 14 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:40,622 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:23:40,622 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:23:40,649 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 11:23:40,649 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 11:23:40,649 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:40,649 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:40,649 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:40,650 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:40,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:40,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,749 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 11:23:40,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 14 11:23:40,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 14, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 11:23:40,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=14, voteGranted=false] 11:23:40,751 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=14, voteGranted=false], my term is 14 11:23:40,751 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=14, voteGranted=false], current voteCount: 1 11:23:40,868 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 11:23:40,869 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-4197593050156667490] but current is [6366852359189618374]. Reason: Cluster member removed, previous status [Down] 11:23:40,929 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:40,929 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,929 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 15 11:23:40,929 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 14 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 15 in "RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 14 - switching to Follower 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=null 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 15, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=15, voteGranted=false] 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 15 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:40,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:40,930 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 11:23:40,931 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 11:23:40,931 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=15, voteGranted=false], my term is 15 11:23:40,931 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=15, voteGranted=false], current voteCount: 1 11:23:40,978 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 11:23:40,978 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} is empty 11:23:40,978 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 11:23:40,988 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:40,988 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 11:23:40,988 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:41,88 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:41,88 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,88 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 16 11:23:41,88 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:41,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:41,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:41,90 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 16 in "RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 15 - updating term 11:23:41,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=null 11:23:41,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 16, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:41,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=16, voteGranted=false] 11:23:41,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=16, voteGranted=false], my term is 16 11:23:41,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=16, voteGranted=false], current voteCount: 1 11:23:41,203 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-4197593050156667490] but current is [6366852359189618374]. Reason: Cluster member removed, previous status [Down] 11:23:41,203 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberRemoved: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:41,209 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:41,209 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 11:23:41,238 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:41,238 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,238 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 17 11:23:41,238 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,239 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:41,239 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:41,240 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:41,240 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 17 in "RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 16 - updating term 11:23:41,240 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=null 11:23:41,240 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 17, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:41,240 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=17, voteGranted=false] 11:23:41,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=17, voteGranted=false], my term is 17 11:23:41,240 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=17, voteGranted=false], current voteCount: 1 11:23:41,253 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING, roles [member-1, dc-default], version [0.0.0] 11:23:41,254 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Welcome from [akka://cluster-test@127.0.0.1:2559] 11:23:41,255 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:41,255 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,255 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,255 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:41,255 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,255 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,255 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 11:23:41,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,368 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 18 11:23:41,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,370 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:41,370 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 11:23:41,370 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 11:23:41,370 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 18 in "RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 17 - updating term 11:23:41,370 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=null 11:23:41,370 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 18, votedFor: null, lastIndex: 1, lastTerm: 4 11:23:41,370 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=18, voteGranted=false] 11:23:41,371 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=18, voteGranted=false], my term is 18 11:23:41,371 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=18, voteGranted=false], current voteCount: 1 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Joining), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Joining) 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 19 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 19 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:41,502 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:41,503 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:23:41,503 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 18 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 19 in "RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 18 - switching to Follower 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=null 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=19, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 19, votedFor: null, lastIndex: 0, lastTerm: 4 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=19, voteGranted=true] 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 19 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:41,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:41,504 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 11:23:41,505 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 11:23:41,505 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=19, voteGranted=true], my term is 19 11:23:41,505 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=19, voteGranted=true], current voteCount: 1 11:23:41,505 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): LastApplied index 0 is behind last index 1 - switching to PreLeader 11:23:41,505 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:41,505 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:23:41,506 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 19 11:23:41,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 11:23:41,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 11:23:41,506 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to PreLeader 11:23:41,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:41,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:41,506 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:23:41,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Replicate message: identifier: null, logIndex: 2, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 11:23:41,506 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:41,506 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:41,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:41,506 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to PreLeader 11:23:41,507 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:41,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:41,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,507 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 11:23:41,507 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 11:23:41,507 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:23:41,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:41,507 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:41,508 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 11:23:41,508 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:41,508 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,508 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:41,508 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc]]] 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=19, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=19, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:41,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=19, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc] 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 4 does not match the current term 19 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 1 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 has no callback 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 2 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 19 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Received ApplyState [identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}]] - lastApplied: 2, lastIndex: 2 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 19 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader)_slicer-id-713: Closing 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 11:23:41,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 11:23:41,510 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from PreLeader to Leader 11:23:41,510 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from PreLeader to Leader 11:23:41,510 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 2 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 2 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@507daffc 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 19 11:23:41,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 11:23:41,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,543 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:23:41,543 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:23:41,543 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:41,543 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 11:23:41,543 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@163d707f 11:23:41,544 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 11:23:41,668 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:41,668 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:41,668 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:41,668 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:41,668 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:41,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, activeMember=true] 11:23:41,888 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:41,889 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 11:23:41,889 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 11:23:41,889 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:41,889 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:41,889 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:41,889 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:41,895 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:41,895 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,895 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:41,895 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:42,275 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:42,275 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,275 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,275 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:42,275 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,909 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:42,909 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:42,909 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:42,909 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:42,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:42,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 11:23:42,909 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 11:23:42,909 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3016be7e, attempting to connect 11:23:42,912 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:23:42,912 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:42,912 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:42,912 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@67c356b8 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49d1ae38 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state DONE 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49d1ae38 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: initiating purge 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 0 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@60af2185 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@69750c11 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860]}, enqueuedTicks=872310714472} 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49d1ae38 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2263/0x00000008016c6af0@ec7bccf 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@69750c11 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE}, enqueuedTicks=874878879011} 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-708: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE}} 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-708: Message size: 459 bytes, total slices to send: 1 11:23:42,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-708: Message does not need to be sliced - sending original message 11:23:42,914 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:42,914 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 11:23:42,914 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 11:23:42,914 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@51d4c2a1 11:23:42,914 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7338906 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5ed11431, splicing queue 11:23:42,914 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7338906 queue spliced 0 messages 11:23:42,914 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.316 ms 11:23:42,916 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=3, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 ready 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 to ready 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct canCommit 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 validated 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct preCommit 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 candidate ready 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - candidate rootPath: / 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - messages: [] 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 prepared 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct commit 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=4, term=19, payload=Simple{size=241}] 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=19, payload=Simple{size=241}] 11:23:42,917 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:42,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 11:23:42,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=19, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=4, term=19, payload=Simple{size=241}]]] 11:23:42,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 submitted to persistence 11:23:42,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:42,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=19, payload=Simple{size=241}] 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=19, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=4, term=19, payload=Simple{size=241}]]] 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=241} 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=19, payload=Simple{size=241}] 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=19, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=4, logLastTerm=19, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 11:23:42,919 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=19, payload=Simple{size=241}] 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 4, nextIndex: 5 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 3 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2275/0x00000008016cf5b0@692bb0ac 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: finished purging inherited transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 4 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 3 to 4 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 4 data Simple{size=241} 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:42,920 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,920 AM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: commit - successfulFromPrevious: [] 11:23:42,920 AM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 11:23:42,920 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1649756259] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 11:23:42,920 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 3 for term 19 11:23:42,920 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:23:42,920 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=4 11:23:42,920 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1649756259] 11:23:42,920 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 11:23:42,920 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1649756259] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:42,920 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 57.46 μs 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 4 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 3 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 4 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 3 to 4 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 4 data Simple{size=241} 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:42,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 11:23:42,922 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 3 for term 19 11:23:42,922 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=4 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 179.7 μs 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860]}, enqueuedTicks=872310714472, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: purge completed 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49d1ae38 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@67c356b8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@30b1f8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE}, enqueuedTicks=874878879011, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], modifications=1, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@4de18508 11:23:42,922 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b: creating a new snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#157887107], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED}: initiating purge 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} allocated sequence 1 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@5c5e048d 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:42,922 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@69750c11 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]}, enqueuedTicks=876257419392} 11:23:42,922 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 11:23:42,922 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} allocated sequence 0 11:23:42,922 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@16424f63 11:23:42,922 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@69750c11 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=876257689418} 11:23:42,927 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,927 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=5, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 11:23:42,927 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 11:23:42,927 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:42,927 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 11:23:42,929 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-712: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, message: Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 11:23:42,929 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=5, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 11:23:42,929 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-712: Message size: 377 bytes, total slices to send: 1 11:23:42,929 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-712: Message does not need to be sliced - sending original message 11:23:42,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 11:23:42,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=876257689418, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 11:23:42,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0} 11:23:42,933 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 11:23:42,933 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,933 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 11:23:42,933 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 11:23:42,933 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Initiating leader transfer 11:23:42,934 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@eb800f2 11:23:42,934 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@eb800f2 11:23:42,934 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Attempting to transfer leadership 11:23:42,934 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 11:23:42,934 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,934 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=4, prevLogTerm=19, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}]]] 11:23:42,934 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:42,934 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=13] 11:23:42,934 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:42,934 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@69750c11 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@39b70ddb, splicing queue 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@69750c11 queue spliced 1 messages 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:42,935 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-183563410] 11:23:42,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:42,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=4, prevLogTerm=19, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}]]] 11:23:42,935 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:23:42,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 11:23:42,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 11:23:42,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 11:23:42,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 11:23:42,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 11:23:42,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=19, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=5, logLastTerm=19, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 5, nextIndex: 6 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 5 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 4 to 5 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@5a8b8392 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: finished purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 4 for term 19 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, matchIndex: 5, lastIndex: 5, isVoting: true 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Follower's log matches - sending ElectionTimeout 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Leader transfer complete 11:23:42,936 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=5 11:23:42,937 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 11:23:42,937 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} 11:23:42,937 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-199214860]}, enqueuedTicks=876257419392, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} 11:23:42,937 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE}: purge completed 11:23:42,937 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49d1ae38 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} 11:23:42,937 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 11:23:42,937 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@67c356b8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 11:23:42,938 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=13] 11:23:42,938 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:42,938 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=5, term=19, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 5 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 5 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 4 to 5 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 4 for term 19 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received TimeoutNow - switching to Candidate 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 20 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=19] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 20 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:42,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,939 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 11:23:42,940 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 11:23:42,940 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 11:23:42,940 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 11:23:42,940 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 11:23:42,940 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=5 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Cluster state: [] 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Candidate in requestVote:RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=19] with higher term appears reachable, updating term. 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Term 20 in "RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=19]" message is greater than leader's term 19 - switching to Follower 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=null 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Leadership transfer in progress - processing RequestVote 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): In requestVote: RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=19] - currentTerm: 20, votedFor: null, lastIndex: 5, lastTerm: 19 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): requestVote returning: RequestVoteReply [term=20, voteGranted=true] 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader) :- Switching from behavior Leader to Follower, election term: 20 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-714: Closing 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 11:23:42,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,942 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Leader to Follower 11:23:42,942 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 11:23:42,942 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Leader to Follower 11:23:42,943 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.SendHeartBeat] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:42,943 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=20, voteGranted=true], my term is 20 11:23:42,943 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=20, voteGranted=true], current voteCount: 1 11:23:42,943 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 11:23:42,943 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:23:42,943 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 20 11:23:42,943 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 11:23:42,943 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 11:23:42,943 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:23:42,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 11:23:42,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:42,944 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 11:23:42,944 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:42,944 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 11:23:42,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is ready - notifying 1 OnShardInitialized callbacks 11:23:42,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3797e1f8] 11:23:42,944 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:42,944 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3797e1f8] 11:23:42,944 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@72754a8b, attempting to connect 11:23:42,944 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:42,944 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,945 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,945 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 11:23:42,945 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:42,945 AM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:42,945 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 11:23:42,945 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Successfully transferred leadership to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges in 11.17 ms 11:23:42,945 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader transfer succeeded - sending PoisonPill 11:23:42,945 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49d1ae38 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3473cf8 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3473cf8 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-199214860], snapshotOnly=true} 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@44d49c6a to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 replayed 1 successful requests 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@284d703b finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1f39a00b 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2523efe8 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5ec3a002, splicing queue 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2523efe8 queue spliced 0 messages 11:23:42,945 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2114551260], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-281828933], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 458.7 μs 11:23:42,946 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,946 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 11:23:42,946 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 11:23:42,946 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 11:23:42,946 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:42,946 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,946 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:23:42,946 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#304471951] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#304471951] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:42,946 AM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:23:42,946 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,946 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 11:23:42,946 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,946 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 11:23:42,947 AM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 11:23:42,947 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,947 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#38937902] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#38937902] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:42,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 5, nextIndex: 6 11:23:42,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 11:23:42,948 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:42,948 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:42,955 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:42,955 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:42,955 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:42,955 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:42,956 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 11:23:42,962 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 11:23:42,963 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 11:23:42,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-715: Closing 11:23:42,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 11:23:42,963 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 11:23:42,964 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:42,964 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:43,967 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6366852359189618374] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:43,981 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:43,981 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:23:43,982 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:43.984] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:43,987 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3495342063657260769] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:23:43,990 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:23:43,991 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:23:43,992 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:23:43.993] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:44,4 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:44,9 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3338867628156560125] 11:23:44,9 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:23:44,11 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:23:44,11 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:23:44,11 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:44,11 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:44,20 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:23:44,20 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:23:44,20 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:23:44,23 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:23:44,27 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8310611098599890829] 11:23:44,27 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:23:44,36 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:44,36 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:23:44,36 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:44,36 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:44,51 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:23:44,56 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7244458044269783025] 11:23:44,56 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:23:44,59 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:23:44,59 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:23:44,59 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:23:44,59 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:23:44,69 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:44,71 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:44,72 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 11:23:44,82 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,83 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 11:23:44,84 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:23:44,92 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:44,101 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 11:23:44,101 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 11:23:44,101 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:23:44,101 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,101 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3a8efb33 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b5e1b9e 11:23:44,101 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@a8e8d56 11:23:44,101 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:44,101 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,102 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709033024101, None)) 11:23:44,102 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033024100,0,0)) 11:23:44,102 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:23:44,102 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b5e1b9e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@617ef7bb 11:23:44,102 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@21e4cb97 11:23:44,102 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 11:23:44,103 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:44,104 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:44,107 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 11:23:44,107 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:44,107 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,108 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,108 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,108 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:44,108 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,108 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 11:23:44,108 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-2050275792 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,108 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#807824676] 11:23:44,108 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-2050275792] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,108 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#807824676] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,108 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 11:23:44,110 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:23:44,110 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:44,110 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:44,111 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:23:44,111 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:23:44,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 11:23:44,111 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 11:23:44,111 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 11:23:44,111 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 11:23:44,111 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 11:23:44,114 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#36048594] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,114 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 11:23:44,116 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 11:23:44,116 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:44,116 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 11:23:44,116 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:23:44,116 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1cec2cb6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@68c232e8 11:23:44,116 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2dc0a194 11:23:44,116 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:44,116 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,116 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709033024116, None)) 11:23:44,116 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033024115,0,0)) 11:23:44,116 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3f7ca350 11:23:44,116 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@68c232e8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7ccbc2a4 11:23:44,116 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 11:23:44,117 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:44,117 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:44,117 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,117 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:44,117 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-268306421] 11:23:44,117 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-268306421] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,117 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 11:23:44,117 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-1405995061 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:44,117 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#960760979] 11:23:44,117 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-1405995061] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,117 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#960760979] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:44,118 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 11:23:44,118 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:44,118 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:44,118 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 11:23:44,118 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 11:23:44,118 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:23:44,118 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:23:44,118 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 11:23:44,118 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,118 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 11:23:44,118 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 11:23:44,167 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:44,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 11:23:44,167 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:44,342 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:23:44,342 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:23:44,343 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 11:23:44,343 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 11:23:44,583 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 11:23:44,583 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,583 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 11:23:44,711 AM [cluster-test-shard-dispatcher-24] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:44,882 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 11:23:44,883 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:44,883 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 11:23:45,23 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:23:45,23 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:23:45,23 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:45,23 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:23:45,23 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:45,23 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:45,23 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,23 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:45,23 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,28 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:45,28 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:23:45,28 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,28 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:45,28 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,28 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,71 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:23:45,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 11:23:45,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:45,172 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 11:23:45,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:45,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:23:45,175 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:23:45,175 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:23:45,175 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:23:45,175 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:45,175 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:23:45,175 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:23:45,175 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 11:23:45,176 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 11:23:45,176 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 11:23:45,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:45,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:45,176 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 11:23:45,176 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:23:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 11:23:45,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:45,177 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 11:23:45,177 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:45,177 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:45,177 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:23:45,177 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 11:23:45,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 11:23:45,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:45,223 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:23:45,224 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} 11:23:45,224 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:45,224 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:45,224 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:45,224 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} shard 1 11:23:45,224 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:23:45,224 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:23:45,224 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 11:23:45,224 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 11:23:45,224 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:23:45,224 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:23:45,224 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:23:45,224 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} allocated sequence 0 11:23:45,225 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:45,225 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15d64a0b] 11:23:45,225 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15d64a0b] 11:23:45,225 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@681a2584, attempting to connect 11:23:45,225 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} is not yet known 11:23:45,225 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=11780, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:23:45,226 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7f26a71d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6775baad 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state OPEN 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6775baad 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6775baad 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6775baad 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@73a071aa 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2cc0242f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@36f1aa25, splicing queue 11:23:45,226 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2cc0242f queue spliced 0 messages 11:23:45,227 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 602.4 μs 11:23:45,235 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:23:45,235 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,235 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 11:23:45,235 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 11:23:45,236 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:45,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 11:23:45,236 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 11:23:45,236 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 11:23:45,236 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 11:23:45,237 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:23:45,237 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], modifications=1} 11:23:45,237 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#36048594] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#36048594] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:45,237 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 11:23:45,237 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:23:45,237 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#36048594] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#36048594] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:45,237 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#1974278706] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1974278706] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:45,238 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 11:23:45,238 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 11:23:45,239 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:45,239 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:45,239 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 11:23:45,239 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:45,239 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:23:45,239 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:45,290 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:45,293 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:45,340 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:45,391 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:45,413 AM [cluster-test-shard-dispatcher-14] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 11:23:45,413 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:23:45,413 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:45,413 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 11:23:45,413 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-720: Closing 11:23:45,413 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:23:45,413 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 11:23:45,413 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 11:23:45,414 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:45,414 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:45,441 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] 11:23:45,441 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], modifications=1} 11:23:45,442 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 11:23:45,442 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 11:23:45,442 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@245b8c7c 11:23:45,442 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:45,442 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7beed006 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}, enqueuedTicks=878777132578} 11:23:45,442 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: false, paused: false 11:23:45,442 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:45,442 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7beed006 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5ff535e3, splicing queue 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7beed006 queue spliced 1 messages 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:45,443 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:45,443 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:45,443 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:45,852 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:45,852 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:45,853 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:45,853 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:45,853 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:45,853 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:46,43 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 11:23:46,873 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:46,873 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:46,873 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:46,873 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:46,873 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:46,873 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:47,292 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:47,292 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:47,292 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:47,293 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:47,293 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:47,293 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:48,312 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:48,312 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:48,312 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:48,312 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:48,312 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:48,312 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:48,732 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:48,732 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:48,733 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:48,733 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:48,733 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:48,733 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:49,752 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:49,752 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:49,752 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:49,752 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:49,753 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:49,753 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:50,173 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:50,173 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:50,174 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:50,174 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:50,174 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:50,174 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:51,193 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:51,193 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:51,193 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:51,193 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:51,193 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:51,193 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:51,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:51,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:51,613 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:51,613 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:51,613 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:51,613 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:52,632 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:52,632 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:52,632 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:52,632 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:52,633 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:52,633 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:53,52 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:53,52 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:53,52 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:53,53 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:53,53 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:53,53 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:54,72 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:54,72 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:54,72 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:54,72 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:54,73 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:54,73 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:54,493 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:54,493 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:54,493 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:54,493 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:54,493 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:54,494 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:55,513 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:55,513 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:55,513 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:55,513 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:55,513 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:55,513 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:55,932 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:55,932 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:55,932 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:55,933 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:55,933 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:55,933 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:56,952 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:56,952 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:56,952 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:56,952 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:56,952 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:56,953 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 11:23:57,372 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 11:23:57,372 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 11:23:57,372 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 11:23:57,372 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:57,373 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:57,373 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 11:23:57,443 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 11:23:57,443 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], modifications=1} 11:23:57,443 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:57,443 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 11:23:57,443 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 11:23:57,443 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:57,445 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:23:57,447 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 11:23:57,449 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 11:23:57,450 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:23:57,451 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 11:23:57,451 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@58fba92c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@101f489 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,451 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4937d3c7 11:23:57,451 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709033037451, None)) 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:23:57,451 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033037450,0,0)) 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,451 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@58f8d68a 11:23:57,451 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@101f489 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@b01c627 11:23:57,452 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:23:57,452 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} allocated sequence 0 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,452 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@5293b350 11:23:57,452 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-33194632] 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-33194632] 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 11:23:57,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,453 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-33194632] 11:23:57,453 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 11:23:57,453 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-1597990233 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:57,453 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 11:23:57,453 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-1274878627] 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:23:57,454 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 11:23:57,454 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 11:23:57,454 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:57,454 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 11:23:57,454 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 11:23:57,534 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 11:23:57,534 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:57,534 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 11:23:57,534 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:57,534 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:57,534 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:23:57,534 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 12357, lastApplied : -1, commitIndex : -1 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-721: Closing 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 11:23:57,535 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 11:23:57,535 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 11:23:57,536 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 11:23:58,392 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,392 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:58,392 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:58,392 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:58,393 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:58,393 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15d64a0b] 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15d64a0b] 11:23:58,393 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@391d38ca, attempting to connect 11:23:58,393 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6775baad instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5da3f9c2 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state SEALED 11:23:58,393 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5da3f9c2 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 0 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@5293b350 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}, enqueuedTicks=890787246097} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5da3f9c2 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} allocated sequence 0 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct canCommit 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@245b8c7c 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 validated 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}, enqueuedTicks=878777132578} 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct preCommit 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5da3f9c2 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 candidate ready 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], modifications=1} 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - messages: [] 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 prepared 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct commit 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3b870c49 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@846ab8c 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 171 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6c4ac6a6 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1fd1b7af, splicing queue 11:23:58,394 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6c4ac6a6 queue spliced 0 messages 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 11:23:58,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.141 ms 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 submitted to persistence 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 initiating direct canCommit 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [WARN] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Store Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1: Conflicting modification for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars. 11:23:58,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to FAILED 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}, enqueuedTicks=878777132578, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@588f0b0a 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,395 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} 11:23:58,396 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED}: initiating purge 11:23:58,396 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onFailure(FrontendReadWriteTransaction.java:441) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.failedCanCommit(SimpleShardDataTreeCohort.java:175) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:862) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:884) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:831) at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:986) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:433) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleCommitLocalTransaction(FrontendReadWriteTransaction.java:524) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:193) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: OptimisticLockFailedException{message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, errorList=[RpcError [message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:846) ... 41 more Caused by: org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction. at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:74) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:181) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:111) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:409) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:368) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:110) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:31) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:838) ... 41 more 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=171} 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 1 11:23:58,396 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:58,396 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 11:23:58,396 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6528d15c 11:23:58,396 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]}, enqueuedTicks=891730755699} 11:23:58,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 0, nextIndex: 1 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 0 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@24ee0e96}, purged=MutableUnsignedLongSet{size=0}} 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}, enqueuedTicks=890787246097, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#852619235] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@5fed44a9 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,399 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} allocated sequence 0 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED}: initiating purge 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} allocated sequence 1 11:23:58,399 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1234/0x00000008013074a0@1cccaf06 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@4d93ec34 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 11:23:58,399 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}, enqueuedTicks=891734608567} 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]}, enqueuedTicks=891734390402} 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 11:23:58,399 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#852619235] 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:58,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,400 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct canCommit 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 validated 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct preCommit 11:23:58,400 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#852619235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,400 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 69.66 μs 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 candidate ready 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - candidate rootPath: / 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - messages: [] 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 prepared 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct commit 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 69 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}]]] 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 submitted to persistence 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 11:23:58,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 0 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3fe4cf18}, purged=MutableUnsignedLongSet{size=0}} 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#342331010] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 11:23:58,401 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 11:23:58,401 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 11:23:58,401 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 11:23:58,402 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 11:23:58,402 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#342331010] 11:23:58,402 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:23:58,402 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 121.9 μs 11:23:58,402 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:58,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}]]] 11:23:58,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 11:23:58,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:23:58,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 11:23:58,402 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 11:23:58,403 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=69} 11:23:58,403 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 11:23:58,403 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 3 11:23:58,403 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:58,403 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 11:23:58,403 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 2, nextIndex: 3 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 1 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@6758be9 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:58,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 11:23:58,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]}, enqueuedTicks=891730755699, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=69} 11:23:58,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE}: purge completed 11:23:58,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5da3f9c2 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} 11:23:58,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6775baad purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: commit - successfulFromPrevious: [] 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false} with Envelope{sessionId=1, txSequence=3, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1467450918], coordinated=false}, enqueuedTicks=891734608567, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} directCommit completed 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED}: initiating purge 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} allocated sequence 1 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 17.68 μs 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@55f95896 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:58,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]}, enqueuedTicks=891739068280} 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}]]] 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 11:23:58,404 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:23:58,405 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:23:58,405 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 11:23:58,405 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 11:23:58,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 11:23:58,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 11:23:58,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@637bce0a 11:23:58,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@637bce0a 11:23:58,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 11:23:58,405 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 11:23:58,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 11:23:58,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:58,405 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:58,405 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:23:58,405 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@51c6d6f1, splicing queue 11:23:58,405 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 11:23:58,405 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ee39c68 queue spliced 2 messages 11:23:58,405 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 11:23:58,405 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 11:23:58,406 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 11:23:58,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1038323173], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 11:23:58,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:23:58,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:23:58,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] 11:23:58,406 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:23:58,406 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 20000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}]]] 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 4 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 2 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 3 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=69} 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 11:23:58,406 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 11:23:58,406 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 19.71 μs 11:23:58,406 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 11:23:58,407 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 11:23:58,407 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 3, nextIndex: 4 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 3 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@47e5f818 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 3, lastIndex: 4, isVoting: true 11:23:58,407 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 11:23:58,407 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} not found in inflight queue, checking pending queue 11:23:58,407 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 11:23:58,407 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]}, enqueuedTicks=891734390402, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 11:23:58,407 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 11:23:58,407 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE}: purge completed 11:23:58,407 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5da3f9c2 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} 11:23:58,407 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:23:58,407 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:23:58,407 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6775baad purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 11:23:58,407 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 3 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 3 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 4 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 3 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 11:23:58,408 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 4, nextIndex: 5 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 4 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@1c430f17 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,408 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} not found in inflight queue, checking pending queue 11:23:58,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1467450918]}, enqueuedTicks=891739068280, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 11:23:58,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE}: purge completed 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 11:23:58,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5da3f9c2 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 4, lastIndex: 4, isVoting: true 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 11:23:58,409 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 11:23:58,409 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 4 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 4 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:23:58,410 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 11:24:00,423 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.018 s 11:24:00,423 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 11:24:00,424 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 11:24:00,424 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-724: Closing 11:24:00,424 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 11:24:00,424 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 11:24:00,424 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 11:24:00,424 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:24:00,424 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:00,425 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1190897427] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1467450918] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1467450918] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1174/0x00000008012cdec8] to Actor[akka://cluster-test/user/$a#-1467450918] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1467450918] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:24:00,425 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:24:00.434] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1094)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:24:00.434] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1094)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:24:00.434] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1094)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:24:00.434] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1094)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:24:00,454 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 11:24:00,457 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 11:24:00,457 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 11:24:00,457 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 11:24:00,457 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 11:24:00,457 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:24:00,458 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:24:01,441 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3338867628156560125] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:24:01,471 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8310611098599890829] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:24:01,473 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:24:01,473 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:24:01,474 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:24:01.475] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:24:01,484 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:24:01,484 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [02/27/2024 11:24:01.486] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:24:01,519 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:24:01,529 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3328769420327103987] 11:24:01,530 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 11:24:01,531 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 11:24:01,531 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 11:24:01,536 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:24:01,536 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:24:01,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 11:24:01,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 11:24:01,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 11:24:01,545 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 11:24:01,549 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5854523187518546769] 11:24:01,549 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 11:24:01,556 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:24:01,556 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 11:24:01,556 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:24:01,556 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:24:01,568 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 11:24:01,576 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7850279959962818353] 11:24:01,576 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 11:24:01,577 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:24:01,577 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 11:24:01,587 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 11:24:01,587 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 11:24:01,587 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:24:01,588 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:24:01,590 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 11:24:01,591 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 11:24:01,594 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:24:01,599 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1637540104] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,600 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:24:01,602 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:24:01,602 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:24:01,602 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 11:24:01,602 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,602 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@11f17764 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4591f704 11:24:01,602 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@17b886a6 11:24:01,602 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,602 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:24:01,602 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,602 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709033041602, None)) 11:24:01,603 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033041601,0,0)) 11:24:01,603 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5511ec04 11:24:01,603 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4591f704 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@25be084b 11:24:01,603 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 11:24:01,603 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:24:01,605 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 11:24:01,606 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421] 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630] 11:24:01,607 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 11:24:01,608 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 11:24:01,608 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 11:24:01,608 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1504337333] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,608 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-929050408 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-199657724] 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-929050408] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-199657724] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1312841241 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,609 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1312841241] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:24:01,610 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:24:01,610 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:24:01,611 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 11:24:01,611 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 11:24:01,611 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:24:01,611 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 11:24:01,611 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:24:01,611 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:24:01,611 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 11:24:01,611 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:24:01,611 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:24:01,612 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 11:24:01,612 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 11:24:01,612 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:24:01,612 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 11:24:01,612 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:24:01,612 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:24:01,612 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 11:24:01,613 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1442931721] 11:24:01,613 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1442931721] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,614 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 11:24:01,614 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:24:01,614 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@ae47aa9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ee7986 11:24:01,614 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@755d7a86 11:24:01,614 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709033041614, None)) 11:24:01,615 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709033041613,0,0)) 11:24:01,615 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ee7986 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@364640d1 11:24:01,615 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 11:24:01,615 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 11:24:01,615 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,615 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:01,615 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 11:24:01,615 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,615 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,615 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2efa7b24 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#2078722491] 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-1568305619] 11:24:01,616 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 11:24:01,616 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 11:24:01,616 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 11:24:01,617 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-146674505] 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#503596740 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-2075360161] 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#2078722491] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-1568305619] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#503596740] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-146674505] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,617 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-2075360161] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,619 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#115592308 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:24:01,619 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#115592308] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:24:01,619 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:24:01,619 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 11:24:01,619 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,619 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 11:24:01,619 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:24:01,619 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,619 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 11:24:01,619 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 11:24:01,619 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 11:24:01,619 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 11:24:01,622 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 11:24:01,623 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 11:24:01,665 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1637540104] 11:24:01,665 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 11:24:01,666 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421] 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:24:01,835 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 11:24:01,836 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:24:01,836 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 11:24:01,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 11:24:01,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 11:24:01,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 11:24:01,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:01,865 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 11:24:01,866 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 11:24:01,866 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 11:24:01,866 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 11:24:01,866 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 11:24:01,866 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 11:24:01,866 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 11:24:02,96 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:24:02,96 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,96 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:24:02,144 AM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:24:02,165 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:24:02,165 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,165 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 11:24:02,214 AM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 11:24:02,415 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:24:02,415 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,415 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:24:02,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:24:02,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,455 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 11:24:02,556 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 11:24:02,556 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 11:24:02,556 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,557 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,564 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,606 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 11:24:02,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:24:02,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,716 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:24:02,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,718 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:24:02,719 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:24:02,719 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:24:02,719 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 11:24:02,719 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:24:02,719 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:24:02,719 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:24:02,719 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:24:02,724 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:24:02,724 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:24:02,724 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:24:02,724 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:24:02,724 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 11:24:02,726 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 11:24:02,726 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,726 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 11:24:02,726 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 11:24:02,726 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,726 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,726 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 11:24:02,726 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:24:02,726 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:24:02,726 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:24:02,727 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:24:02,727 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,727 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 11:24:02,728 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 11:24:02,728 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 11:24:02,729 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 11:24:02,729 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,729 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 11:24:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 11:24:02,729 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:24:02,729 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:24:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 11:24:02,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 11:24:02,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 11:24:02,775 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1637540104] 11:24:02,775 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 11:24:02,775 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630] 11:24:02,775 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@39cc65e6: creating a new transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:24:02,776 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:24:02,776 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:24:02,776 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:24:02,776 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1504337333] 11:24:02,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:24:02,776 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 11:24:02,776 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:24:02,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:24:02,776 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:24:02,776 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:24:02,776 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:24:02,776 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:24:02,776 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:24:02,776 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1504337333] 11:24:02,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:24:02,776 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 11:24:02,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:24:02,776 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:24:02,776 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:24:02,776 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 11:24:02,776 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@39cc65e6 readied transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 11:24:02,776 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@12b7d55e 11:24:02,776 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 11:24:02,776 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1229/0x0000000801306990@5be89856 11:24:02,776 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 11:24:02,777 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@75c07995, attempting to connect 11:24:02,777 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5fe06200, attempting to connect 11:24:02,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 11:24:02,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=10090, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:24:02,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 11:24:02,778 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=11390, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:24:02,779 AM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:24:02,779 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d340127 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@188a277f 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@188a277f 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@605aee5c 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10bfefee transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=896111710861} 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-728: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE}} 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-728: Message size: 392 bytes, total slices to send: 1 11:24:02,784 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-728: Message does not need to be sliced - sending original message 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@39cc65e6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3fe6013d 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2314bbea superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1a5feadb, splicing queue 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2314bbea queue spliced 0 messages 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 927.1 μs 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13dd86c4 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@537dbdd3 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@537dbdd3 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2225/0x00000008016a8470@30dca6e2 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@174a97d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=896111806744} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-728: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE}} 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-728: Message size: 397 bytes, total slices to send: 1 11:24:02,785 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-728: Message does not need to be sliced - sending original message 11:24:02,786 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@39cc65e6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:24:02,786 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 11:24:02,786 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4f6e0163 11:24:02,786 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@52c9f996 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@20d16211, splicing queue 11:24:02,786 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@52c9f996 queue spliced 0 messages 11:24:02,786 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 761.7 μs 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,787 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,788 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 11:24:02,788 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 11:24:02,788 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 11:24:02,788 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,788 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 11:24:02,788 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 11:24:02,788 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 11:24:02,788 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 11:24:02,788 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=896111710861, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 11:24:02,788 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 11:24:02,789 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 11:24:02,789 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=896111806744, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 11:24:02,789 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@66e61aff 11:24:02,789 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@4a573e58 11:24:02,789 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:24:02,789 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10bfefee transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896124311981} 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013041f8@5562356c 11:24:02,789 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 11:24:02,789 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1242/0x0000000801303d08@172ea71f 11:24:02,789 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:24:02,789 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@174a97d transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896124514806} 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 11:24:02,791 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 11:24:02,791 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 11:24:02,792 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 11:24:02,792 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896124514806, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 11:24:02,792 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:24:02,792 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 11:24:02,792 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896124311981, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 11:24:02,792 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 11:24:02,794 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 11:24:02,794 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@329f3578 11:24:02,794 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:24:02,794 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 11:24:02,794 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@329f3578 11:24:02,794 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:24:02,794 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10bfefee transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896129735846} 11:24:02,794 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 11:24:02,794 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@7612af12 11:24:02,795 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:24:02,795 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 11:24:02,795 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307030@7612af12 11:24:02,795 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:24:02,795 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@174a97d transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896129937720} 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:24:02,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 11:24:02,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 11:24:02,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 11:24:02,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 11:24:02,795 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 11:24:02,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 11:24:02,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:24:02,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 11:24:02,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 11:24:02,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:24:02,796 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 11:24:02,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:24:02,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 11:24:02,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=176} 11:24:02,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 11:24:02,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 11:24:02,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:24:02,797 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 11:24:02,797 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 11:24:02,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 11:24:02,797 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@54f4c183}, purged=MutableUnsignedLongSet{size=0}} 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7c93cd7a}, purged=MutableUnsignedLongSet{size=0}} 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 11:24:02,797 AM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 11:24:02,798 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 11:24:02,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 11:24:02,798 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#2061237173] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 11:24:02,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1105162104] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 11:24:02,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:24:02,798 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#2061237173] 11:24:02,798 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:24:02,798 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1105162104] 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#2061237173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1105162104] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 42.10 μs 11:24:02,798 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 76.01 μs 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4587476f}, purged=MutableUnsignedLongSet{size=0}} 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#992037945] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 11:24:02,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@25a8b820}, purged=MutableUnsignedLongSet{size=0}} 11:24:02,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:24:02,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1532519208] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 11:24:02,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#992037945] 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896129735846, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d340127 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 48.89 μs 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6193d7f4 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#992037945] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@65ac5f1e 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1532519208] 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10bfefee transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896135377835} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896129937720, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13dd86c4 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 11:24:02,800 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1532519208] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@6e176af0 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 124.0 μs 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307268@11c43cb1 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:24:02,801 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@174a97d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896135841406} 11:24:02,801 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@39cc65e6: creating a new snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:24:02,801 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:24:02,801 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 11:24:02,801 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@36e4576a 11:24:02,801 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10bfefee transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=896136492891} 11:24:02,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,802 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,802 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 11:24:02,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 11:24:02,802 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 11:24:02,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 11:24:02,802 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:24:02,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 11:24:02,802 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 11:24:02,802 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-726: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 11:24:02,803 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-726: Message size: 332 bytes, total slices to send: 1 11:24:02,803 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-726: Message does not need to be sliced - sending original message 11:24:02,803 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 11:24:02,803 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 11:24:02,803 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 11:24:02,803 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=896136492891, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 11:24:02,803 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 11:24:02,803 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 11:24:02,803 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 11:24:02,803 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1193/0x00000008012d6268@20bd4d32 11:24:02,803 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@174a97d transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=896138689042} 11:24:02,804 AM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 11:24:02,804 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-727: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 11:24:02,804 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-727: Message size: 337 bytes, total slices to send: 1 11:24:02,804 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-727: Message does not need to be sliced - sending original message 11:24:02,805 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 11:24:02,805 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1107537459], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=896138689042, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 11:24:02,805 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 11:24:02,805 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:24:02,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:02,805 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 11:24:02,805 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 11:24:02,805 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 11:24:02,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 11:24:02,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 11:24:02,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69f1951c 11:24:02,805 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78d7b3e1 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69f1951c 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78d7b3e1 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 11:24:02,806 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 11:24:02,806 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 11:24:02,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 11:24:02,807 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 11:24:02,807 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 11:24:02,807 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@174a97d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5ddf997b, splicing queue 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 11:24:02,807 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@174a97d queue spliced 1 messages 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 2 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1182213630], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1504337333] 11:24:02,807 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10bfefee superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@7ebcea3e, splicing queue 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 11:24:02,807 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10bfefee queue spliced 1 messages 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 1 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 11:24:02,807 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1504337333] 11:24:02,807 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 11:24:02,807 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@2cb24260 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,807 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2229/0x00000008016ab8d0@1a4efdfa 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896135377835, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@188a277f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d340127 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 11:24:02,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1107537459]}, enqueuedTicks=896135841406, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@537dbdd3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 11:24:02,808 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13dd86c4 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 11:24:02,808 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:24:02,808 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 11:24:02,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:24:02,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 11:24:03,225 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 11:24:03,225 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#848059421]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 11:24:03,226 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 11:24:03,226 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-731: Closing 11:24:03,226 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:24:03,226 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 11:24:03,226 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-732: Closing 11:24:03,226 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:24:03,226 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:24:03,227 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:24:03,227 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 11:24:03,227 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 11:24:03,227 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:24:03,227 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:24:03,228 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:24:03,228 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:24:03,228 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1637540104] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1637540104] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. [WARN] [02/27/2024 11:24:03.232] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:24:03.232] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:24:03.232] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [02/27/2024 11:24:03.232] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:24:03,248 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 11:24:03,248 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 11:24:03,248 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 11:24:03,248 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 11:24:03,248 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 11:24:03,248 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:24:03,248 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 11:24:03,249 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 11:24:03,249 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:24:03,249 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:24:04,246 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3328769420327103987] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:24:04,266 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:24:04,266 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 11:24:04,266 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5854523187518546769] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 11:24:04,267 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:24:04.269] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 11:24:04,277 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 11:24:04,277 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 11:24:04,280 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [02/27/2024 11:24:04.280] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config